-- Logs begin at Mon 2022-08-01 14:36:41 UTC, end at Mon 2022-08-01 16:22:42 UTC. -- Aug 01 15:18:55.631957 dv-cinder-847536-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 01 15:18:55.640994 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 01 15:18:55.643067 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 1 15:18:55 2022] *** Aug 01 15:18:55.643067 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 01 15:18:55.643067 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 01 15:18:55.643067 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: nodename: dv-cinder-847536-3-iscsi Aug 01 15:18:55.643067 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: machine: x86_64 Aug 01 15:18:55.643067 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: clock source: unix Aug 01 15:18:55.643067 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: pcre jit disabled Aug 01 15:18:55.643067 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: detected number of CPU cores: 4 Aug 01 15:18:55.643067 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: current working directory: / Aug 01 15:18:55.643067 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: detected binary path: /usr/bin/uwsgi-core Aug 01 15:18:55.643067 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: your processes number limit is 47680 Aug 01 15:18:55.643650 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: your memory page size is 4096 bytes Aug 01 15:18:55.643650 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: detected max file descriptor number: 2048 Aug 01 15:18:55.643650 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: lock engine: pthread robust mutexes Aug 01 15:18:55.643650 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: thunder lock: enabled Aug 01 15:18:55.643650 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 01 15:18:55.643650 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 01 15:18:55.682133 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: Python main interpreter initialized at 0x563bde23c9c0 Aug 01 15:18:55.682133 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: python threads support enabled Aug 01 15:18:55.682133 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: your server socket listen backlog is limited to 100 connections Aug 01 15:18:55.682133 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: your mercy for graceful operations on workers is 90 seconds Aug 01 15:18:55.682541 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: mapped 403077 bytes (393 KB) for 2 cores Aug 01 15:18:55.682541 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: *** Operational MODE: preforking *** Aug 01 15:18:55.682831 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: *** uWSGI is running in multiple interpreter mode *** Aug 01 15:18:55.682831 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: spawned uWSGI master process (pid: 80336) Aug 01 15:18:55.683043 dv-cinder-847536-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 01 15:18:55.685931 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: spawned uWSGI worker 1 (pid: 80348, cores: 1) Aug 01 15:18:55.685931 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: spawned uWSGI worker 2 (pid: 80349, cores: 1) Aug 01 15:18:55.685931 dv-cinder-847536-3-iscsi devstack@keystone.service[80336]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 01 15:19:01.215307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 15:19:01.215307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: from cryptography.utils import int_from_bytes Aug 01 15:19:01.216104 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 15:19:01.216104 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: from cryptography.utils import int_from_bytes Aug 01 15:19:01.216628 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 15:19:01.216628 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: from cryptography.utils import int_from_bytes Aug 01 15:19:01.217085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 15:19:01.217085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: from cryptography.utils import int_from_bytes Aug 01 15:19:01.266005 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 15:19:01.266208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 15:19:01.266386 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] command line args: [] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 15:19:01.266666 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 15:19:01.266848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ================================================================================ {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 15:19:01.267114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] admin_token = **** {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.267363 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] config_dir = [] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.267556 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.267677 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 15:19:01.267765 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] config_source = [] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.267881 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 15:19:01.267967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] debug = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.268078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] command line args: [] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 15:19:01.268164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.268348 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 15:19:01.268472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.268552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ================================================================================ {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 15:19:01.268624 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.268765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] admin_token = **** {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.268876 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.269013 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] config_dir = [] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.269139 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.269250 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.269356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] list_limit = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.269459 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] config_source = [] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.269564 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] log_config_append = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.269667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] debug = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.269773 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.269906 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.270047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] log_dir = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.270147 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.270254 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] log_file = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.270372 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.270485 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.270611 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.270695 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.270819 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.270897 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.271013 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] list_limit = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.271097 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.271180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] log_config_append = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.271253 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.271327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.271405 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.271526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] log_dir = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.271526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] log_file = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.271768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.271866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.271989 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.272066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.272192 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.272270 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.272405 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.272483 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.272558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.272674 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.272751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.272877 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.272955 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.273079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.273180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.273267 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.273456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.273537 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.273687 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.273943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.274037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.83/identity {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.274165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] publish_errors = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.274250 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.274405 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.274533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.274644 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.274735 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.274834 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.274913 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.275086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.275327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.275412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.83/identity {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.275564 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] standard_threads = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.275644 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] publish_errors = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.275798 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.275892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.276035 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.276119 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.276198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.276275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.276377 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] use_journal = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.276505 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.276580 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] use_json = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.276682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.276760 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] use_stderr = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.276889 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] use_syslog = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.276973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] standard_threads = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.277125 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.277210 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.277374 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.277457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.277579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.277666 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.277745 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] use_journal = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.277905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.277997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] use_json = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.278114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] use_stderr = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.278192 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.278293 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] use_syslog = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.278379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.278533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:19:01.278611 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.278799 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.278954 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.279081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.279151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.279290 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.279376 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.279533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.279618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.279788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.279874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.280023 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.280102 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.280274 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.280360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.280516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.280608 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.280760 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.280838 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.281009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.281087 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.281246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.281327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.281484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.281604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.281734 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.281818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.281988 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.282078 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.282238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.282318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.282486 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.282565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.282727 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.282808 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.282985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.283064 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.283236 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.283314 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.283476 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.283563 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.283712 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.283792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.283949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.284033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.284188 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.284274 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.284440 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.284526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.284691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.284770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.284925 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.285003 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.285156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.285240 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.285388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.285475 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.285627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.285707 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.285869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.285976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.286140 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.286231 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.286373 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.286457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.286608 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.286691 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.286846 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.286937 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.287080 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.287159 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.287317 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.287395 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.287551 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.287703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.287785 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.287858 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.288012 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.288091 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.288252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.288328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.288482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.288559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.288714 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.288791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.288945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.289031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.289176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.289253 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.289410 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.289492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.289657 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.289747 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.289935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.290018 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.290147 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.290258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.290378 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.290496 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.290598 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.290696 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.290839 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.290946 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.291076 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.291189 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.291296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.291383 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.291534 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.291653 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.291773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.291865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.292008 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.292088 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.292242 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.292334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.292477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.292554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.292713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.292839 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.292946 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.293069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.293709 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.293791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.293973 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.294058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.294216 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.294344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.294478 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.294557 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.294715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.294806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.294952 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.295036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.295192 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.295274 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.295430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.295517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.295666 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.295748 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.295919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.296000 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.296160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.296241 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.296399 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.296483 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.296638 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.296720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.296877 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.296960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.297114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.297192 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.297349 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.297429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.297587 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.297669 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.297822 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.297944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.298099 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.298179 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.298340 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.298428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.298578 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.298656 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.298816 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.298894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.299053 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.299134 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.299290 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.299373 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.299528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.299616 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.299764 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.299841 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.299986 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.300068 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.300220 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.300300 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.300455 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.300545 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.300692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.300773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.300925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.301005 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.301165 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.301250 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.301400 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.301486 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.301631 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.301750 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.301881 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.301994 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.302128 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.302211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.302363 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.302449 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.302608 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.302693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.302842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] auth.external = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.302923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.303076 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.303193 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.303317 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.303397 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.303579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.303660 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.303822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] auth.password = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.303906 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.304056 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] auth.token = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.304171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.304295 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] auth.external = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.304375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.304531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.304620 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.304782 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.304863 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.305017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.305114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.305274 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] auth.password = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.305361 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.305511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] auth.token = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.305592 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.305745 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.305832 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.306008 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] credential.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.306092 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.306258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.306337 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.306493 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.306586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.306746 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.306825 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.306988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.307069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.307221 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.307309 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.307458 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.307539 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] credential.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.307707 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.307786 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.307961 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.308049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.308199 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.308279 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.308443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.308522 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.308697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.308819 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.309521 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.309621 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.309802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.309916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.310077 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.310169 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.310325 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.310429 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.310570 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] federation.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.310651 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.310816 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.310918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.311072 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.311180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.311320 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.311432 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.311556 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.311697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.311880 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.312003 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.312123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.312250 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] federation.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.312377 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.312510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.312632 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.312755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.312881 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.313002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.313125 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.313265 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.313265 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.313496 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.313496 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.313722 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.313845 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.314025 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.314025 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.314259 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.314259 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.314404 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.314517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.314737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.314863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.314983 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.315110 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.315240 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.315352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.315431 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.315504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.315576 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.315648 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.315720 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.315795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.315869 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.316016 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.316098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.316251 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.316332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.316497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.316578 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.316744 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.316824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.316994 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.317075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.317229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.317310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.317467 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.317547 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.317703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.317782 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.317974 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.318078 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.318237 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.318319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.318468 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.318547 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.318708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.318790 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.318942 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.319025 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.319179 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.319518 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.319687 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.319769 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.319929 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.320010 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.320181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.320262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.320417 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.320500 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.320647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.320726 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.320886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.320967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.321129 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.321213 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.321368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.321447 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.321615 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.321694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.321866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.322004 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.322108 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.322210 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.322337 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.322445 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.322581 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.322690 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.322821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.322929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.323066 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.323175 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.323285 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.323525 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.323761 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.324002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.324243 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.324323 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.324529 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.324610 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.324781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.324868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.325014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.325094 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.325255 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.325336 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.325492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.325573 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.325732 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.325810 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.325993 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.326248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.326501 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.326580 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.326742 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.326821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.326998 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.327080 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.327242 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.327473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.327747 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.327949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.328201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.328283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.328467 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.328707 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.328786 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.328961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.329069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.329213 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.329329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.329463 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.329586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.329702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.329840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.329960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.330110 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.330190 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.330347 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.330439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.330618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.330710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.330825 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.330906 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.331066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.331156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.331302 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.331390 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.331555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.331636 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.331805 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.331941 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.332055 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.332199 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.332304 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.332438 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.332555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.332691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.332773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.332934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.333015 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.333176 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.333257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.333418 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.333500 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.333659 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.333740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.333956 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.334085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.334177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.334338 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.334419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.334591 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.334672 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.334837 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.334926 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.335098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.335179 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.335335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.335415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.335572 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.335663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.335813 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.335894 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.336055 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.336135 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.336297 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.336385 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.336536 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.336617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.336777 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.336859 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.337011 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.337100 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] resource.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.337258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.337368 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.337505 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] resource.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.337587 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.337741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.337831 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.338017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.338099 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.338260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.338339 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.338497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.338589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.338749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.338831 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.338989 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.339077 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.339232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.339388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.339465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.339537 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] role.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.339701 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.339782 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] role.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.339933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.340023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] role.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.340208 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] role.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.340295 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.340445 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.340527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.340694 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.340772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.340943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.341025 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.341190 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.341270 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.341438 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.341520 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.341689 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.341770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.341959 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.342044 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.342198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.342281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.342449 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.342528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.342686 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.342767 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.342932 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.343014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.343178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.343258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.343429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.343610 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.343699 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.343853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.344094 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.344175 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.344337 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.344418 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.344587 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.344667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.344839 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.344920 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.345078 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.345159 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.345332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.345413 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.345574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.345655 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.345812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.345907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.346072 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.346155 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.346309 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.346391 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.346545 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.346625 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.346794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.346875 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.347031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.347112 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.347282 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.347371 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.347521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.347603 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.347759 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.347840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.347993 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.348083 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.348235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] token.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.348338 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.348468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.348546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.348709 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] token.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.348798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.348946 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.349026 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.349196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.349276 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.349440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.349542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.349680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.349765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.349945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.350030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.350194 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.350289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.350442 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.350524 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.350679 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.350758 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.350920 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.351013 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.351157 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.351237 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.351400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.351481 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.351638 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.351726 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.351871 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.351951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.352116 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.352197 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.352346 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.352433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.352591 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.352672 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.connection = **** {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.352890 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.352981 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.353139 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.353220 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.connection = **** {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.353382 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.353473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.353627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.353710 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.353865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.353958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.354136 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.354229 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.354376 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.354460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.354617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.354698 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.354857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.354938 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.355099 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.355181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.355346 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.355427 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.355587 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.356196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.356373 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.356467 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.356610 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.356691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.356856 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.356936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.357092 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.357174 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.357333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.357415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.357564 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.357644 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.357726 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 15:19:01.357802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.358060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:19:01.358233 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 15:19:01.387525 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.387753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.387861 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.387983 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.388071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.388282 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.396371 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.396606 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.426647 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.426872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.427810 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.428044 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.480375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.481515 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.531811 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.532033 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.533077 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.533344 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.597934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.598103 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.598186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.598303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.598412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.598527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.598613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.598755 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.598879 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.598963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.599103 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.599213 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.622540 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.622776 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.622987 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.623133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.623250 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.623373 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.623497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.623619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.623753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.623870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.623997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.624117 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.624117 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.624362 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.634972 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.635277 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.648203 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.648437 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.648527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.648666 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.648752 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.648905 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.649005 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.649155 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.649249 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.649361 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.649446 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.649576 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.649659 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.649796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:19:01.666485 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.666693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.666794 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 15:19:01.666908 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 15:19:01.667027 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.667226 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 15:19:01.667313 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.667437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 15:19:01.667521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.667710 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.667802 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.667974 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.668065 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.668260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.668360 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.668521 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.677940 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.678220 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.678319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.678491 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.678589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.678770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.678869 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.679070 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.686682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.686833 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.691554 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.691675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.691841 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.691936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.692120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.692210 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.692402 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.692492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.692681 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.692772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.692941 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.693038 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.693206 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.693294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.693478 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.693566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.693745 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.693843 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.694052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.694142 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.694568 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.694659 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.718236 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.718477 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.719939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.720044 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.720044 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.720199 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.724654 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.724753 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.724890 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.724985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.725162 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.725287 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.731854 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.731970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.732141 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.732233 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.736344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.736633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.736928 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.737211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.737488 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.737618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.737797 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.737916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.738201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.738290 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.738485 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.738579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.738758 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.739041 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.739318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.739595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.754913 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.755200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.756117 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.756396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.759144 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.760354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.762360 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.762656 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.763575 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.763877 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.767050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.768283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.770310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.771513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.773471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.773770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.774081 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.774377 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.774664 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.774794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.774941 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.775078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.775267 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.775556 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.775836 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.776135 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.791518 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.791795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.792064 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.792333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.792750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.793037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.793361 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.793629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.799322 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.800578 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.800864 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.801230 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.802145 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.802419 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.803991 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.804271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.804577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.804860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.805148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.805278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.805518 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.805802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.806122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.806413 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.817183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.817461 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.817741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.818045 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.818328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.818493 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.818634 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.818765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.818893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.819032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.819560 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.819840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.820112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.820387 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.834798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.836216 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.837738 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.839228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.841234 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.842735 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.843775 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.845288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.845454 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.845629 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.845934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.846809 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.847116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.847423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.852578 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.854120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.854325 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.856092 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.857103 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.857392 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.857679 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.857997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.859633 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.860543 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.860543 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.860543 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.866059 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.866350 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.866642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.869178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.869466 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.869745 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.874053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.874331 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.874611 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.874892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.875169 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.875459 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.875748 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.876024 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.876394 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.876678 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.876961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.877209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:19:01.877336 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.877493 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.877629 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.877756 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.877880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.878052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.878345 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.878629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.878907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.879178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.879552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.879837 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.880125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.880400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.880679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.880944 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:19:01.911718 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x563bde23c9c0 pid: 80349 (default app) Aug 01 15:19:01.915022 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x563bde23c9c0 pid: 80348 (default app) Aug 01 15:19:01.915181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45b456-c624-4460-a33f-e354045272dd None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:01.915470 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45b456-c624-4460-a33f-e354045272dd None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:01.915745 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45b456-c624-4460-a33f-e354045272dd None None] PATH_INFO: `/v3` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:01.920580 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1/1] 10.222.0.83 () {50 vars in 791 bytes} [Mon Aug 1 15:19:01 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:09.628272 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8c3ce-6e39-4580-a0d3-6d52d4070641 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:09.628844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8c3ce-6e39-4580-a0d3-6d52d4070641 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:09.628844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8c3ce-6e39-4580-a0d3-6d52d4070641 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:09.633239 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1/2] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:19:09.640051 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d0d7d-b32e-4553-9909-5aefd52288d0 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:09.640515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d0d7d-b32e-4553-9909-5aefd52288d0 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:09.640605 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d0d7d-b32e-4553-9909-5aefd52288d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:09.717060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: INFO dbcounter [None req-3d4d0d7d-b32e-4553-9909-5aefd52288d0 None None] Registered counter for database keystone Aug 01 15:19:09.736216 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG oslo_db.sqlalchemy.engines [None req-3d4d0d7d-b32e-4553-9909-5aefd52288d0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80349) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 15:19:09.742767 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writer thread running {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 15:19:09.927689 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG passlib.handlers.bcrypt [None req-3d4d0d7d-b32e-4553-9909-5aefd52288d0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80349) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 15:19:09.928018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG passlib.handlers.bcrypt [None req-3d4d0d7d-b32e-4553-9909-5aefd52288d0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80349) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 15:19:09.988464 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3d4d0d7d-b32e-4553-9909-5aefd52288d0 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:09.990194 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3d4d0d7d-b32e-4553-9909-5aefd52288d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:10.129088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 2/3] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 490 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 15:19:10.266810 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8c3ce-6e39-4580-a0d3-6d52d4070641 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:10.340266 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: INFO dbcounter [None req-b1c8c3ce-6e39-4580-a0d3-6d52d4070641 None None] Registered counter for database keystone Aug 01 15:19:10.365375 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG oslo_db.sqlalchemy.engines [None req-b1c8c3ce-6e39-4580-a0d3-6d52d4070641 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80348) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 15:19:10.372236 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writer thread running {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 15:19:10.493869 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8c3ce-6e39-4580-a0d3-6d52d4070641 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:10.498796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667004f5-4f99-4bc0-9b44-5adf14794d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:10.499451 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667004f5-4f99-4bc0-9b44-5adf14794d24 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:10.499778 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667004f5-4f99-4bc0-9b44-5adf14794d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:10.500117 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667004f5-4f99-4bc0-9b44-5adf14794d24 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:10.508339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667004f5-4f99-4bc0-9b44-5adf14794d24 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:10.560528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: WARNING keystone.common.rbac_enforcer.enforcer [None req-667004f5-4f99-4bc0-9b44-5adf14794d24 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 15:19:10.560802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667004f5-4f99-4bc0-9b44-5adf14794d24 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:10.571004 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 2/4] 10.222.0.83 () {58 vars in 1324 bytes} [Mon Aug 1 15:19:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 15:19:10.577349 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d0d7d-b32e-4553-9909-5aefd52288d0 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:10.592046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d0d7d-b32e-4553-9909-5aefd52288d0 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:10.596809 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6ff44-5470-4774-b687-ff313b573d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:10.597954 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6ff44-5470-4774-b687-ff313b573d06 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:10.598240 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6ff44-5470-4774-b687-ff313b573d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:10.598513 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6ff44-5470-4774-b687-ff313b573d06 admin admin] PATH_INFO: `/v3/projects/6c0cb293301d4dcdbc4412b1e11a284f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:10.599998 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6ff44-5470-4774-b687-ff313b573d06 admin admin] RBAC: Authorizing `identity:get_project(project_id=6c0cb293301d4dcdbc4412b1e11a284f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:10.651466 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: WARNING keystone.common.rbac_enforcer.enforcer [None req-94f6ff44-5470-4774-b687-ff313b573d06 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 15:19:10.652089 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6ff44-5470-4774-b687-ff313b573d06 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:10.657028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 3/5] 10.222.0.83 () {56 vars in 1207 bytes} [Mon Aug 1 15:19:10 2022] GET /identity/v3/projects/6c0cb293301d4dcdbc4412b1e11a284f => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:12.724489 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f1d03-7649-44de-adb6-f2afbb9f4263 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:12.725002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f1d03-7649-44de-adb6-f2afbb9f4263 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:12.725109 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f1d03-7649-44de-adb6-f2afbb9f4263 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:12.729153 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 3/6] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:12.736229 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a067849-7b67-468d-9934-b3313582eccc None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:12.736695 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a067849-7b67-468d-9934-b3313582eccc None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:12.736793 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a067849-7b67-468d-9934-b3313582eccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:12.757349 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3a067849-7b67-468d-9934-b3313582eccc None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:12.758910 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3a067849-7b67-468d-9934-b3313582eccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:12.769489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 4/7] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 01 15:19:12.791325 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f1d03-7649-44de-adb6-f2afbb9f4263 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:12.803976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f1d03-7649-44de-adb6-f2afbb9f4263 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:12.808221 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c465b1-2f52-4cad-8399-e982fc848db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:12.809419 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c465b1-2f52-4cad-8399-e982fc848db7 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:12.809703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c465b1-2f52-4cad-8399-e982fc848db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:12.810088 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c465b1-2f52-4cad-8399-e982fc848db7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:12.817750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c465b1-2f52-4cad-8399-e982fc848db7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:12.819828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c465b1-2f52-4cad-8399-e982fc848db7 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:12.828853 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 4/8] 10.222.0.83 () {58 vars in 1324 bytes} [Mon Aug 1 15:19:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:19:12.834636 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a067849-7b67-468d-9934-b3313582eccc None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:12.844263 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a067849-7b67-468d-9934-b3313582eccc None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:12.848586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304cfa3-70c2-4ac7-a04e-e1088f691227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:12.849219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304cfa3-70c2-4ac7-a04e-e1088f691227 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:12.849500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304cfa3-70c2-4ac7-a04e-e1088f691227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:12.849774 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304cfa3-70c2-4ac7-a04e-e1088f691227 admin admin] PATH_INFO: `/v3/users/01b3cc95b90b46cf85a902fea655d1ea` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:12.851161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a304cfa3-70c2-4ac7-a04e-e1088f691227 admin admin] RBAC: Authorizing `identity:get_user(user_id=01b3cc95b90b46cf85a902fea655d1ea)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:12.852996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a304cfa3-70c2-4ac7-a04e-e1088f691227 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:12.854587 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 5/9] 10.222.0.83 () {56 vars in 1198 bytes} [Mon Aug 1 15:19:12 2022] GET /identity/v3/users/01b3cc95b90b46cf85a902fea655d1ea => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:14.940016 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a185482-c79a-4b35-b542-f9fca9e63e2c None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:14.940558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a185482-c79a-4b35-b542-f9fca9e63e2c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:14.940733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a185482-c79a-4b35-b542-f9fca9e63e2c None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:14.942055 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 5/10] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:19:14.947437 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceeeaac-2ac0-4f95-8928-be75a1369cc0 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:14.947840 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceeeaac-2ac0-4f95-8928-be75a1369cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:14.947991 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceeeaac-2ac0-4f95-8928-be75a1369cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:14.968356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-5ceeeaac-2ac0-4f95-8928-be75a1369cc0 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:14.969558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5ceeeaac-2ac0-4f95-8928-be75a1369cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:15.023563 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39973953-c2b4-4794-a49b-959ef7013880 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:15.023856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39973953-c2b4-4794-a49b-959ef7013880 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:15.024134 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39973953-c2b4-4794-a49b-959ef7013880 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:15.025513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 6/11] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:19:15.030906 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5764e7-c4b4-4b1b-a28b-12516b7d819b None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:15.031197 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5764e7-c4b4-4b1b-a28b-12516b7d819b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:15.031519 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5764e7-c4b4-4b1b-a28b-12516b7d819b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:15.051396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 6/12] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:15.074831 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceeeaac-2ac0-4f95-8928-be75a1369cc0 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:15.075049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG passlib.handlers.bcrypt [None req-bf5764e7-c4b4-4b1b-a28b-12516b7d819b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80348) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 15:19:15.075344 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG passlib.handlers.bcrypt [None req-bf5764e7-c4b4-4b1b-a28b-12516b7d819b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80348) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 15:19:15.098623 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:15.121799 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-bf5764e7-c4b4-4b1b-a28b-12516b7d819b None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:15.123467 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-bf5764e7-c4b4-4b1b-a28b-12516b7d819b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:15.181133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceeeaac-2ac0-4f95-8928-be75a1369cc0 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:15.216808 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac239a-ba67-406c-873c-97b2022baf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:15.217387 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac239a-ba67-406c-873c-97b2022baf79 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:15.217673 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac239a-ba67-406c-873c-97b2022baf79 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:15.217989 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac239a-ba67-406c-873c-97b2022baf79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:15.224699 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 7/13] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:19:15.229164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac239a-ba67-406c-873c-97b2022baf79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:15.238778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-2aac239a-ba67-406c-873c-97b2022baf79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:19:15.238778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:15.238778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:15.238778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:15.238778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:15.238778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:15.238778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:15.238778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:15.238778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:15.238778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:15.238778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:15.238778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:15.238778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:15.238778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:15.238778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:15.238778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:15.239872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:15.239872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:15.239872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:15.239872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:15.239872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:15.239872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:15.239872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:15.239872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:15.239872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:15.239872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:15.239872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:15.239872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:15.239872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:15.239872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:15.239872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:15.239872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:15.241218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:15.241218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:15.241218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:15.241218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:15.241218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:15.241218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:15.241218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:15.241218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:15.241218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:15.241218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:15.241218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:19:15.241218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:19:15.241218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 7/14] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:19:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:19:15.245781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5764e7-c4b4-4b1b-a28b-12516b7d819b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:15.246908 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac239a-ba67-406c-873c-97b2022baf79 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:15.330520 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:15.349552 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac239a-ba67-406c-873c-97b2022baf79 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:15.350018 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5764e7-c4b4-4b1b-a28b-12516b7d819b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:15.352358 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:15.384791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aed84f-f6dd-4c90-88ae-f0c2d0120e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:15.385445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aed84f-f6dd-4c90-88ae-f0c2d0120e46 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:15.385727 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aed84f-f6dd-4c90-88ae-f0c2d0120e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:15.386036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aed84f-f6dd-4c90-88ae-f0c2d0120e46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:15.388770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef4b28-6b21-49bc-bcd4-fc2464753ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:15.389703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef4b28-6b21-49bc-bcd4-fc2464753ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:15.390020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef4b28-6b21-49bc-bcd4-fc2464753ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:15.390298 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef4b28-6b21-49bc-bcd4-fc2464753ed1 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:15.391192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef4b28-6b21-49bc-bcd4-fc2464753ed1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:15.391503 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef4b28-6b21-49bc-bcd4-fc2464753ed1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:15.392959 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef4b28-6b21-49bc-bcd4-fc2464753ed1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:15.405392 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 8/15] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:15.410979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef4b28-6b21-49bc-bcd4-fc2464753ed1 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:15.455220 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aed84f-f6dd-4c90-88ae-f0c2d0120e46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:15.456719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aed84f-f6dd-4c90-88ae-f0c2d0120e46 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:15.516025 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef4b28-6b21-49bc-bcd4-fc2464753ed1 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:15.542570 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:15.550867 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a2e2d-468c-4501-bbc9-f95b2166de59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:15.551748 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a2e2d-468c-4501-bbc9-f95b2166de59 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:15.552031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a2e2d-468c-4501-bbc9-f95b2166de59 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:15.552305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a2e2d-468c-4501-bbc9-f95b2166de59 None admin] PATH_INFO: `/v3/users/01b3cc95b90b46cf85a902fea655d1ea` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:15.553852 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a2e2d-468c-4501-bbc9-f95b2166de59 None admin] RBAC: Authorizing `identity:get_user(user_id=01b3cc95b90b46cf85a902fea655d1ea)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:15.555336 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a2e2d-468c-4501-bbc9-f95b2166de59 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:15.556954 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 9/16] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:15 2022] GET /identity/v3/users/01b3cc95b90b46cf85a902fea655d1ea => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:15.562376 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a2e2d-468c-4501-bbc9-f95b2166de59 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:15.564627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 8/17] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:19:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:19:15.570220 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aed84f-f6dd-4c90-88ae-f0c2d0120e46 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:15.599411 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:15.676501 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a2e2d-468c-4501-bbc9-f95b2166de59 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:15.713302 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa27df3-a4a1-4828-8479-eb1ee40851ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:15.713998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa27df3-a4a1-4828-8479-eb1ee40851ad None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:15.714278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa27df3-a4a1-4828-8479-eb1ee40851ad None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:15.715256 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa27df3-a4a1-4828-8479-eb1ee40851ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:15.716381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa27df3-a4a1-4828-8479-eb1ee40851ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:15.717848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa27df3-a4a1-4828-8479-eb1ee40851ad None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:15.719495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 10/18] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:19:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:15.725404 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa27df3-a4a1-4828-8479-eb1ee40851ad None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:15.783258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aed84f-f6dd-4c90-88ae-f0c2d0120e46 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:15.818747 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f4a7c-0104-4f9c-b719-4bc391c52a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:15.819666 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f4a7c-0104-4f9c-b719-4bc391c52a98 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:15.819950 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f4a7c-0104-4f9c-b719-4bc391c52a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:15.820227 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f4a7c-0104-4f9c-b719-4bc391c52a98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:15.828965 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa27df3-a4a1-4828-8479-eb1ee40851ad None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:15.830729 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:15.831938 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f4a7c-0104-4f9c-b719-4bc391c52a98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:15.833649 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f4a7c-0104-4f9c-b719-4bc391c52a98 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:15.849028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-650f4a7c-0104-4f9c-b719-4bc391c52a98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:19:15.849028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:15.849028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:15.849028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:15.849028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:15.849028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:15.849028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:15.849028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:15.849028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:15.849028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:15.849028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:15.849028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:15.849028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:19:15.849028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:19:15.849028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:19:15.850034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:19:15.850034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:15.850034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:15.850034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:15.850034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:15.850034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:15.850034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:15.850034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:15.850034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:15.850034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:15.850034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:15.850034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:15.850034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:15.850034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:15.850034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:15.850034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:15.850034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:15.850875 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:19:15.850875 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:19:15.850875 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:19:15.850875 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:19:15.850875 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 9/19] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:19:15.856136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f4a7c-0104-4f9c-b719-4bc391c52a98 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:15.865004 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59026cae-e9cb-4446-953c-2226cf5261ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:15.865633 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59026cae-e9cb-4446-953c-2226cf5261ec None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:15.865938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59026cae-e9cb-4446-953c-2226cf5261ec None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:15.866216 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59026cae-e9cb-4446-953c-2226cf5261ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:15.867094 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59026cae-e9cb-4446-953c-2226cf5261ec None admin] RBAC: Adding query filter params (role.id=7bdfa43dc1d046f7965ffe52b74bc58f, scope.domain.id=default, user.id=01b3cc95b90b46cf85a902fea655d1ea) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:15.867333 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59026cae-e9cb-4446-953c-2226cf5261ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:15.868784 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59026cae-e9cb-4446-953c-2226cf5261ec None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:15.878097 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 11/20] 10.222.0.83 () {56 vars in 1322 bytes} [Mon Aug 1 15:19:15 2022] GET /identity/v3/role_assignments?user.id=01b3cc95b90b46cf85a902fea655d1ea&scope.domain.id=default&role.id=7bdfa43dc1d046f7965ffe52b74bc58f => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:15.893108 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:15.967760 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f4a7c-0104-4f9c-b719-4bc391c52a98 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:16.004797 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d9fe5-04cf-4a3f-a581-35062b507597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:16.005432 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d9fe5-04cf-4a3f-a581-35062b507597 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:16.005716 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d9fe5-04cf-4a3f-a581-35062b507597 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:16.006026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d9fe5-04cf-4a3f-a581-35062b507597 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:16.006737 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d9fe5-04cf-4a3f-a581-35062b507597 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:16.007041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d9fe5-04cf-4a3f-a581-35062b507597 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:16.008460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d9fe5-04cf-4a3f-a581-35062b507597 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:16.027646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 10/21] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:18.323239 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b55df-6397-4443-b268-15e7351fe426 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:18.323723 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b55df-6397-4443-b268-15e7351fe426 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:18.323723 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b55df-6397-4443-b268-15e7351fe426 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:18.325918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 12/22] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:18.332730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50d17f-243d-48a0-a5ca-6f2dc1142906 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:18.333169 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50d17f-243d-48a0-a5ca-6f2dc1142906 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:18.333360 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50d17f-243d-48a0-a5ca-6f2dc1142906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:18.358653 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6a50d17f-243d-48a0-a5ca-6f2dc1142906 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:18.359932 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6a50d17f-243d-48a0-a5ca-6f2dc1142906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:18.423577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1444c-483c-4f7e-a641-b72cf8bd5f19 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:18.423872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1444c-483c-4f7e-a641-b72cf8bd5f19 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:18.424132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1444c-483c-4f7e-a641-b72cf8bd5f19 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:18.425732 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 13/23] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:18.431345 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353ab26-bfa3-402b-8784-33ef07e235b5 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:18.431623 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353ab26-bfa3-402b-8784-33ef07e235b5 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:18.431901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353ab26-bfa3-402b-8784-33ef07e235b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:18.462809 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e353ab26-bfa3-402b-8784-33ef07e235b5 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:18.464034 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e353ab26-bfa3-402b-8784-33ef07e235b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:18.484903 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 11/24] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:18.573111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50d17f-243d-48a0-a5ca-6f2dc1142906 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:18.578517 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 14/25] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:19:18.599415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:18.617322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353ab26-bfa3-402b-8784-33ef07e235b5 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:18.713052 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50d17f-243d-48a0-a5ca-6f2dc1142906 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:18.774189 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a240789-2b30-4e1b-959e-d022d3adaba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:18.777583 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a240789-2b30-4e1b-959e-d022d3adaba5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:18.779322 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a240789-2b30-4e1b-959e-d022d3adaba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:18.779893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a240789-2b30-4e1b-959e-d022d3adaba5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:18.806262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:18.806599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a240789-2b30-4e1b-959e-d022d3adaba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:18.812457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-5a240789-2b30-4e1b-959e-d022d3adaba5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:19:18.812457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:18.812457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:18.812457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:18.812457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:18.812457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:18.812457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:18.812457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:18.812457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:18.812457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:18.812457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:18.812457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:18.812457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:18.812457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:18.812457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:18.812457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:18.813334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:18.813334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:18.813334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:18.813334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:18.813334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:18.813334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:18.813334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:18.813334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:18.813334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:18.813334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:18.813334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:18.813334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:18.813334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:18.813334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:18.813334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:18.813334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:18.814118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:18.814118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:18.814118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:18.814118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:18.814118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:18.814118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:18.814118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:18.814118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:18.814118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:18.814118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:18.814118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:19:18.814118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:19:18.814118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 12/26] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:19:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:19:18.819864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353ab26-bfa3-402b-8784-33ef07e235b5 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:18.825501 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4152a1-0b44-487a-9cbf-b492b87e55cb None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:18.825791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4152a1-0b44-487a-9cbf-b492b87e55cb None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:18.826173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4152a1-0b44-487a-9cbf-b492b87e55cb None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:18.830102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 13/27] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:18 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:18.831111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4152a1-0b44-487a-9cbf-b492b87e55cb None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:18.865641 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47ebf9-3d71-45c9-ae0d-8bd90db05de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:18.866338 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47ebf9-3d71-45c9-ae0d-8bd90db05de7 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:18.866782 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47ebf9-3d71-45c9-ae0d-8bd90db05de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:18.867200 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47ebf9-3d71-45c9-ae0d-8bd90db05de7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:18.888830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47ebf9-3d71-45c9-ae0d-8bd90db05de7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:18.890777 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47ebf9-3d71-45c9-ae0d-8bd90db05de7 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:18.911228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-1f47ebf9-3d71-45c9-ae0d-8bd90db05de7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:19:18.911228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:18.911228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:18.911228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:18.911228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:18.911228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:18.911228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:18.911228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:18.911228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:18.911228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:18.911228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:18.911228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:18.911228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:19:18.911228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:19:18.911228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:19:18.912104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:19:18.912104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:18.912104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:18.912104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:18.912104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:18.912104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:18.912104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:18.912104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:18.912104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:18.912104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:18.912104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:18.912104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:18.912104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:18.912104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:18.912104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:18.912104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:18.912104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:18.912888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:19:18.912888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:19:18.912888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:19:18.912888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:19:18.914867 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 15/28] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:19:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:19:18.916683 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500491a-7c15-40fb-8c81-cbdccb8e5a3b None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:18.917171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500491a-7c15-40fb-8c81-cbdccb8e5a3b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:18.917724 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500491a-7c15-40fb-8c81-cbdccb8e5a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:18.937833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-5500491a-7c15-40fb-8c81-cbdccb8e5a3b None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:18.939490 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5500491a-7c15-40fb-8c81-cbdccb8e5a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:18.988862 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:19.011458 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4152a1-0b44-487a-9cbf-b492b87e55cb None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:19.046133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 16/29] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:19.047990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500491a-7c15-40fb-8c81-cbdccb8e5a3b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:19.069622 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3877a-fbbd-4edc-91ce-13e84a46ba93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:19.070289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3877a-fbbd-4edc-91ce-13e84a46ba93 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:19.070645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3877a-fbbd-4edc-91ce-13e84a46ba93 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:19.070927 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3877a-fbbd-4edc-91ce-13e84a46ba93 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:19.071712 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3877a-fbbd-4edc-91ce-13e84a46ba93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:19.072010 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3877a-fbbd-4edc-91ce-13e84a46ba93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:19.073428 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3877a-fbbd-4edc-91ce-13e84a46ba93 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:19.087311 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 14/30] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:19.089934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd813f-e146-43d4-982c-f7adaec1fcfe None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:19.089934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd813f-e146-43d4-982c-f7adaec1fcfe None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:19.089934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd813f-e146-43d4-982c-f7adaec1fcfe None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:19.090962 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 15/31] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:19.091652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd813f-e146-43d4-982c-f7adaec1fcfe None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:19.158372 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:19.183618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500491a-7c15-40fb-8c81-cbdccb8e5a3b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:19.209218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd813f-e146-43d4-982c-f7adaec1fcfe None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:19.249308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753291cb-d503-4d9e-9553-dd66df71444e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:19.250041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753291cb-d503-4d9e-9553-dd66df71444e None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:19.250337 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753291cb-d503-4d9e-9553-dd66df71444e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:19.250649 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753291cb-d503-4d9e-9553-dd66df71444e None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:19.251457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753291cb-d503-4d9e-9553-dd66df71444e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:19.252947 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753291cb-d503-4d9e-9553-dd66df71444e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:19.333408 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd593f5-8797-49e1-974d-b5fae187f615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:19.334294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd593f5-8797-49e1-974d-b5fae187f615 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:19.334740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd593f5-8797-49e1-974d-b5fae187f615 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:19.335186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd593f5-8797-49e1-974d-b5fae187f615 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:19.336081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd593f5-8797-49e1-974d-b5fae187f615 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:19.336525 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd593f5-8797-49e1-974d-b5fae187f615 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:19.338176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd593f5-8797-49e1-974d-b5fae187f615 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:19.357582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 17/32] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:19.358705 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd593f5-8797-49e1-974d-b5fae187f615 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:19.395310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 16/33] 10.222.0.83 () {60 vars in 1131 bytes} [Mon Aug 1 15:19:19 2022] POST /identity/v3/roles => generated 219 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:19.397104 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d14c2-7b0c-40d0-8648-b955927b80bb None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:19.397408 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d14c2-7b0c-40d0-8648-b955927b80bb None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:19.397690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d14c2-7b0c-40d0-8648-b955927b80bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:19.413071 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:19.421852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-215d14c2-7b0c-40d0-8648-b955927b80bb None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:19.423532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-215d14c2-7b0c-40d0-8648-b955927b80bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:19.478870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd593f5-8797-49e1-974d-b5fae187f615 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:19.506982 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 17/34] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:19.507669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d14c2-7b0c-40d0-8648-b955927b80bb None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:19.520075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69795e-7de8-4d9b-b98b-bb3e644594dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:19.521109 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69795e-7de8-4d9b-b98b-bb3e644594dc None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:19.521608 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69795e-7de8-4d9b-b98b-bb3e644594dc None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:19.522113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69795e-7de8-4d9b-b98b-bb3e644594dc None admin] PATH_INFO: `/v3/users/01b3cc95b90b46cf85a902fea655d1ea` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:19.523666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69795e-7de8-4d9b-b98b-bb3e644594dc None admin] RBAC: Authorizing `identity:get_user(user_id=01b3cc95b90b46cf85a902fea655d1ea)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:19.525641 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69795e-7de8-4d9b-b98b-bb3e644594dc None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:19.527852 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 18/35] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:19 2022] GET /identity/v3/users/01b3cc95b90b46cf85a902fea655d1ea => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:19.533478 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69795e-7de8-4d9b-b98b-bb3e644594dc None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:19.544032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:19.628131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d14c2-7b0c-40d0-8648-b955927b80bb None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:19.651079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69795e-7de8-4d9b-b98b-bb3e644594dc None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:19.674105 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65780a-aa58-4bda-b1e8-72e66d1b7a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:19.674105 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65780a-aa58-4bda-b1e8-72e66d1b7a12 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:19.674105 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65780a-aa58-4bda-b1e8-72e66d1b7a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:19.674105 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65780a-aa58-4bda-b1e8-72e66d1b7a12 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:19.674513 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65780a-aa58-4bda-b1e8-72e66d1b7a12 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:19.679371 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65780a-aa58-4bda-b1e8-72e66d1b7a12 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:19.684255 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:19.717356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 18/36] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Aug 1 15:19:19 2022] POST /identity/v3/projects => generated 301 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:19.718258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65780a-aa58-4bda-b1e8-72e66d1b7a12 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:19.738318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d5e5c-3acd-4967-8ba5-141a19c1b84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:19.739133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d5e5c-3acd-4967-8ba5-141a19c1b84c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:19.739422 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d5e5c-3acd-4967-8ba5-141a19c1b84c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:19.739715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d5e5c-3acd-4967-8ba5-141a19c1b84c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:19.742298 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d5e5c-3acd-4967-8ba5-141a19c1b84c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:19.743811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d5e5c-3acd-4967-8ba5-141a19c1b84c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:19.746248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 19/37] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:19:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:19.755156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d5e5c-3acd-4967-8ba5-141a19c1b84c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:19.884748 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:19.913104 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65780a-aa58-4bda-b1e8-72e66d1b7a12 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:19.921217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d5e5c-3acd-4967-8ba5-141a19c1b84c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:19.956484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b5414-0af1-4f17-a402-9e450bfd4275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:19.957144 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b5414-0af1-4f17-a402-9e450bfd4275 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:19.959051 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b5414-0af1-4f17-a402-9e450bfd4275 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:19.959051 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b5414-0af1-4f17-a402-9e450bfd4275 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:19.959051 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b5414-0af1-4f17-a402-9e450bfd4275 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:19.959862 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b5414-0af1-4f17-a402-9e450bfd4275 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:19.963670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb040f5-c34c-4be4-9470-9622aee99db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:19.964914 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb040f5-c34c-4be4-9470-9622aee99db7 None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:19.965452 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb040f5-c34c-4be4-9470-9622aee99db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:19.965871 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb040f5-c34c-4be4-9470-9622aee99db7 None admin] PATH_INFO: `/v3/domains/default/users/01b3cc95b90b46cf85a902fea655d1ea/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:19.968337 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb040f5-c34c-4be4-9470-9622aee99db7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=01b3cc95b90b46cf85a902fea655d1ea, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:19.969982 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb040f5-c34c-4be4-9470-9622aee99db7 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:19.977391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 19/38] 10.222.0.83 () {60 vars in 1131 bytes} [Mon Aug 1 15:19:19 2022] POST /identity/v3/roles => generated 225 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:19.981179 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a0ac4-8329-42ec-a376-6545bd2d12f3 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:19.981401 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a0ac4-8329-42ec-a376-6545bd2d12f3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:19.981801 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a0ac4-8329-42ec-a376-6545bd2d12f3 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:19.983657 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 20/39] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:19.990979 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179710a4-dde2-49bd-8b41-909f67aed111 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:19.991321 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179710a4-dde2-49bd-8b41-909f67aed111 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:19.991606 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179710a4-dde2-49bd-8b41-909f67aed111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:20.016218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-179710a4-dde2-49bd-8b41-909f67aed111 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:20.017449 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-179710a4-dde2-49bd-8b41-909f67aed111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:20.084927 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:20.111633 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 20/40] 10.222.0.83 () {58 vars in 1361 bytes} [Mon Aug 1 15:19:19 2022] PUT /identity/v3/domains/default/users/01b3cc95b90b46cf85a902fea655d1ea/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:20.116049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 21/41] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:20.143395 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb040f5-c34c-4be4-9470-9622aee99db7 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:20.194785 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f528d3-39a4-4920-abc5-000b2ea6be3d None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:20.195446 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f528d3-39a4-4920-abc5-000b2ea6be3d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:20.195878 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f528d3-39a4-4920-abc5-000b2ea6be3d None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:20.197318 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 22/42] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:19:20.205098 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b82df-13d6-427f-9ae4-fd7444a5f830 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:20.205669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b82df-13d6-427f-9ae4-fd7444a5f830 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:20.206168 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b82df-13d6-427f-9ae4-fd7444a5f830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:20.226741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-421b82df-13d6-427f-9ae4-fd7444a5f830 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:20.228000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-421b82df-13d6-427f-9ae4-fd7444a5f830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:20.279886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb040f5-c34c-4be4-9470-9622aee99db7 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:20.321577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325f05e-8ebd-40bf-9001-1c8614bb9721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:20.322503 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325f05e-8ebd-40bf-9001-1c8614bb9721 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:20.322918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325f05e-8ebd-40bf-9001-1c8614bb9721 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:20.323058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325f05e-8ebd-40bf-9001-1c8614bb9721 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:20.324313 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325f05e-8ebd-40bf-9001-1c8614bb9721 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:20.325992 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325f05e-8ebd-40bf-9001-1c8614bb9721 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:20.329956 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 23/43] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:19:20.330723 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 21/44] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:19:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:20.336043 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b82df-13d6-427f-9ae4-fd7444a5f830 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:20.353327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325f05e-8ebd-40bf-9001-1c8614bb9721 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:20.358093 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:20.467694 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b82df-13d6-427f-9ae4-fd7444a5f830 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:20.480069 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:20.484927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325f05e-8ebd-40bf-9001-1c8614bb9721 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:20.523490 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73936f2d-4ef6-44b4-8e84-2e47bac3ce24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:20.524131 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73936f2d-4ef6-44b4-8e84-2e47bac3ce24 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:20.524434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73936f2d-4ef6-44b4-8e84-2e47bac3ce24 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:20.524698 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73936f2d-4ef6-44b4-8e84-2e47bac3ce24 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:20.525421 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73936f2d-4ef6-44b4-8e84-2e47bac3ce24 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:20.526273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9235066-b28b-4eed-937e-dcec9c8253bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:20.526952 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73936f2d-4ef6-44b4-8e84-2e47bac3ce24 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:20.527296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9235066-b28b-4eed-937e-dcec9c8253bf None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:20.527816 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9235066-b28b-4eed-937e-dcec9c8253bf None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:20.528359 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9235066-b28b-4eed-937e-dcec9c8253bf None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:20.529801 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9235066-b28b-4eed-937e-dcec9c8253bf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:20.532660 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9235066-b28b-4eed-937e-dcec9c8253bf None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:20.554064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 22/45] 10.222.0.83 () {60 vars in 1131 bytes} [Mon Aug 1 15:19:20 2022] POST /identity/v3/roles => generated 223 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:19:20.588204 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 24/46] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Aug 1 15:19:20 2022] POST /identity/v3/projects => generated 312 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:19:22.517175 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4286b-3063-4df4-ae74-dbd6c87e4d56 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:22.517689 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4286b-3063-4df4-ae74-dbd6c87e4d56 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:22.517770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4286b-3063-4df4-ae74-dbd6c87e4d56 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:22.519242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 23/47] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:22.524163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89f7cf-c09d-4425-8ecd-9ad519a91a09 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:22.524547 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89f7cf-c09d-4425-8ecd-9ad519a91a09 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:22.524870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89f7cf-c09d-4425-8ecd-9ad519a91a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:22.543821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3d89f7cf-c09d-4425-8ecd-9ad519a91a09 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:22.545034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3d89f7cf-c09d-4425-8ecd-9ad519a91a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:22.616400 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 25/48] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:22.638949 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4286b-3063-4df4-ae74-dbd6c87e4d56 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:22.704074 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:22.745632 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4286b-3063-4df4-ae74-dbd6c87e4d56 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:22.779104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ebb9dd-17f2-45fe-8700-369d70482361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:22.779680 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebb9dd-17f2-45fe-8700-369d70482361 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:22.779966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebb9dd-17f2-45fe-8700-369d70482361 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:22.780246 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebb9dd-17f2-45fe-8700-369d70482361 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:22.791839 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ebb9dd-17f2-45fe-8700-369d70482361 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:22.799124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-03ebb9dd-17f2-45fe-8700-369d70482361 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:19:22.799124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:22.799124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:22.799124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:22.799124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:22.799124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:22.799124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:22.799124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:22.799124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:22.799124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:22.799124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:22.799124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:22.799124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:22.799124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:22.799124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:22.799124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:22.799967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:22.799967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:22.799967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:22.799967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:22.799967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:22.799967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:22.799967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:22.799967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:22.799967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:22.799967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:22.799967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:22.799967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:22.799967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:22.799967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:22.799967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:22.799967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:22.800985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:22.800985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:22.800985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:22.800985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:22.800985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:22.800985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:22.800985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:22.800985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:22.800985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:22.800985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:22.800985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:19:22.800985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:19:22.800985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 24/49] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:19:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:22.806203 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89f7cf-c09d-4425-8ecd-9ad519a91a09 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:22.898267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:22.905906 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da19863-0b85-4d5a-a4c6-3d76c5bcf581 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:22.906212 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da19863-0b85-4d5a-a4c6-3d76c5bcf581 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:22.906495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da19863-0b85-4d5a-a4c6-3d76c5bcf581 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:22.907850 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 25/50] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:19:22.910407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89f7cf-c09d-4425-8ecd-9ad519a91a09 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:22.913795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7dd47-841a-4af1-af1d-30f7898b30cb None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:22.914116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7dd47-841a-4af1-af1d-30f7898b30cb None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:22.914403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7dd47-841a-4af1-af1d-30f7898b30cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:22.933008 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-24b7dd47-841a-4af1-af1d-30f7898b30cb None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:22.934585 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-24b7dd47-841a-4af1-af1d-30f7898b30cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:22.945735 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f91ed-117d-49cc-a174-5600a0c4ee73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:22.946403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f91ed-117d-49cc-a174-5600a0c4ee73 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:22.946689 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f91ed-117d-49cc-a174-5600a0c4ee73 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:22.946972 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f91ed-117d-49cc-a174-5600a0c4ee73 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:22.947695 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f91ed-117d-49cc-a174-5600a0c4ee73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:22.948009 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f91ed-117d-49cc-a174-5600a0c4ee73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:22.950116 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f91ed-117d-49cc-a174-5600a0c4ee73 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:22.958685 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 26/51] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:22.964374 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f91ed-117d-49cc-a174-5600a0c4ee73 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:23.003059 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 26/52] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:19:23.026052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7dd47-841a-4af1-af1d-30f7898b30cb None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:23.065421 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f91ed-117d-49cc-a174-5600a0c4ee73 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:23.082574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:23.100911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308de07a-91cd-4e02-a787-5d05fea1d373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:23.101572 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308de07a-91cd-4e02-a787-5d05fea1d373 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:23.101856 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308de07a-91cd-4e02-a787-5d05fea1d373 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:23.102166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308de07a-91cd-4e02-a787-5d05fea1d373 None admin] PATH_INFO: `/v3/users/01b3cc95b90b46cf85a902fea655d1ea` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:23.103371 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308de07a-91cd-4e02-a787-5d05fea1d373 None admin] RBAC: Authorizing `identity:get_user(user_id=01b3cc95b90b46cf85a902fea655d1ea)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:23.104865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308de07a-91cd-4e02-a787-5d05fea1d373 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:23.106528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 27/53] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:22 2022] GET /identity/v3/users/01b3cc95b90b46cf85a902fea655d1ea => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:23.111903 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308de07a-91cd-4e02-a787-5d05fea1d373 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:23.133564 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:23.135481 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7dd47-841a-4af1-af1d-30f7898b30cb None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:23.170968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b0793-e453-446e-9dde-d2f04e2debfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:23.171623 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b0793-e453-446e-9dde-d2f04e2debfe None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:23.171910 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b0793-e453-446e-9dde-d2f04e2debfe None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:23.172191 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b0793-e453-446e-9dde-d2f04e2debfe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:23.173367 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b0793-e453-446e-9dde-d2f04e2debfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:23.174880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b0793-e453-446e-9dde-d2f04e2debfe None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:23.176688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 27/54] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:19:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:23.182216 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b0793-e453-446e-9dde-d2f04e2debfe None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:23.213044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308de07a-91cd-4e02-a787-5d05fea1d373 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:23.248111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff65d8c-39c9-4056-8d8b-c221226c5608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:23.248761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff65d8c-39c9-4056-8d8b-c221226c5608 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:23.249043 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff65d8c-39c9-4056-8d8b-c221226c5608 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:23.249319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff65d8c-39c9-4056-8d8b-c221226c5608 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:23.250494 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff65d8c-39c9-4056-8d8b-c221226c5608 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:23.251939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff65d8c-39c9-4056-8d8b-c221226c5608 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:23.253561 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 28/55] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:19:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:23.259189 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff65d8c-39c9-4056-8d8b-c221226c5608 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:23.283030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b0793-e453-446e-9dde-d2f04e2debfe None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:23.303592 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:23.319174 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba118e1-6a5a-47af-9176-0af8aacacccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:23.320068 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba118e1-6a5a-47af-9176-0af8aacacccc None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:23.320356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba118e1-6a5a-47af-9176-0af8aacacccc None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:23.320630 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba118e1-6a5a-47af-9176-0af8aacacccc None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:23.321313 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba118e1-6a5a-47af-9176-0af8aacacccc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:23.322793 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba118e1-6a5a-47af-9176-0af8aacacccc None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:23.348316 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:23.357693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 28/56] 10.222.0.83 () {60 vars in 1140 bytes} [Mon Aug 1 15:19:23 2022] POST /identity/v3/projects => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:19:23.363217 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff65d8c-39c9-4056-8d8b-c221226c5608 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:23.401197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d719bd0b-ffbe-493d-8354-42b4c380e0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:23.401876 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719bd0b-ffbe-493d-8354-42b4c380e0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:23.402191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719bd0b-ffbe-493d-8354-42b4c380e0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:23.402467 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719bd0b-ffbe-493d-8354-42b4c380e0a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:23.403307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d719bd0b-ffbe-493d-8354-42b4c380e0a6 None admin] RBAC: Adding query filter params (role.id=7bdfa43dc1d046f7965ffe52b74bc58f, scope.domain.id=default, user.id=01b3cc95b90b46cf85a902fea655d1ea) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:23.403618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d719bd0b-ffbe-493d-8354-42b4c380e0a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:23.405133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d719bd0b-ffbe-493d-8354-42b4c380e0a6 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:23.413304 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 29/57] 10.222.0.83 () {56 vars in 1322 bytes} [Mon Aug 1 15:19:23 2022] GET /identity/v3/role_assignments?user.id=01b3cc95b90b46cf85a902fea655d1ea&scope.domain.id=default&role.id=7bdfa43dc1d046f7965ffe52b74bc58f => generated 530 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:25.417644 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4632ae5-91b9-4630-9388-b12438cd6c5d None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:25.418181 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4632ae5-91b9-4630-9388-b12438cd6c5d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:25.418291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4632ae5-91b9-4630-9388-b12438cd6c5d None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:25.419655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 29/58] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:25.425193 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4e356-04e7-41da-88a5-012a4abf3d5b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:25.425589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4e356-04e7-41da-88a5-012a4abf3d5b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:25.425765 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4e356-04e7-41da-88a5-012a4abf3d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:25.443536 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c8c4e356-04e7-41da-88a5-012a4abf3d5b None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:25.444985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c8c4e356-04e7-41da-88a5-012a4abf3d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:25.512032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 30/59] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:25.536538 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4632ae5-91b9-4630-9388-b12438cd6c5d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:25.653241 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4632ae5-91b9-4630-9388-b12438cd6c5d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:25.687298 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a4a76-be36-41fc-970c-04a179376c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:25.687871 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a4a76-be36-41fc-970c-04a179376c1e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:25.688155 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a4a76-be36-41fc-970c-04a179376c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:25.688431 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a4a76-be36-41fc-970c-04a179376c1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:25.689585 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a4a76-be36-41fc-970c-04a179376c1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:25.691075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a4a76-be36-41fc-970c-04a179376c1e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:25.692692 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 30/60] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:19:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:25.698855 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4e356-04e7-41da-88a5-012a4abf3d5b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:25.782392 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:25.801284 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4e356-04e7-41da-88a5-012a4abf3d5b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:25.835620 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d686bc7-f775-4b0a-8a1f-bb2580e24dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:25.836249 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d686bc7-f775-4b0a-8a1f-bb2580e24dbb None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:25.836538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d686bc7-f775-4b0a-8a1f-bb2580e24dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:25.836820 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d686bc7-f775-4b0a-8a1f-bb2580e24dbb None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:25.837504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d686bc7-f775-4b0a-8a1f-bb2580e24dbb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:25.839014 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d686bc7-f775-4b0a-8a1f-bb2580e24dbb None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:25.865528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 31/61] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Aug 1 15:19:25 2022] POST /identity/v3/users => generated 275 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:29.306435 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786faf4a-661d-41bb-b709-433b67e9268b None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:29.306931 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786faf4a-661d-41bb-b709-433b67e9268b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:29.306931 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786faf4a-661d-41bb-b709-433b67e9268b None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:29.309121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 31/62] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:29.314215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d2e2b-5b01-4521-adfe-68d2c8075450 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:29.315021 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d2e2b-5b01-4521-adfe-68d2c8075450 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:29.315517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d2e2b-5b01-4521-adfe-68d2c8075450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:29.342954 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-398d2e2b-5b01-4521-adfe-68d2c8075450 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:29.344510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-398d2e2b-5b01-4521-adfe-68d2c8075450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:29.442652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 32/63] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:29.475367 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bf464-8b60-4e43-9755-f0e3eaa06a1c None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:29.475700 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bf464-8b60-4e43-9755-f0e3eaa06a1c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:29.475936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bf464-8b60-4e43-9755-f0e3eaa06a1c None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:29.476693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d2e2b-5b01-4521-adfe-68d2c8075450 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:29.477694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 32/64] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:29.497829 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab90a9-5a8b-488c-85ec-5134c31769c1 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:29.498162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab90a9-5a8b-488c-85ec-5134c31769c1 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:29.498422 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab90a9-5a8b-488c-85ec-5134c31769c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:29.527012 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-9aab90a9-5a8b-488c-85ec-5134c31769c1 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:29.528761 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9aab90a9-5a8b-488c-85ec-5134c31769c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:29.566219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:29.573354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:29.607593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d2e2b-5b01-4521-adfe-68d2c8075450 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:29.615755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 33/65] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:19:29.617121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d77f0-66f8-459c-bc41-1fff79477d3a None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:29.617506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d77f0-66f8-459c-bc41-1fff79477d3a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:29.617838 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d77f0-66f8-459c-bc41-1fff79477d3a None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:29.619218 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 34/66] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:29.624493 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20618c99-0d68-4156-85a5-15b983d3ca90 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:29.625006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20618c99-0d68-4156-85a5-15b983d3ca90 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:29.625433 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20618c99-0d68-4156-85a5-15b983d3ca90 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:29.626879 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 35/67] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:29.628114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0bfe3-3d7c-4e11-a11a-b17ca7dd754a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:29.628626 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0bfe3-3d7c-4e11-a11a-b17ca7dd754a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:29.629113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0bfe3-3d7c-4e11-a11a-b17ca7dd754a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:29.648570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-74f0bfe3-3d7c-4e11-a11a-b17ca7dd754a None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:29.649787 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-74f0bfe3-3d7c-4e11-a11a-b17ca7dd754a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:29.653847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fcada4-e9a8-411b-851e-246be9130ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:29.654672 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcada4-e9a8-411b-851e-246be9130ccd None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:29.654993 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcada4-e9a8-411b-851e-246be9130ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:29.655232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcada4-e9a8-411b-851e-246be9130ccd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:29.666219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fcada4-e9a8-411b-851e-246be9130ccd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:29.672327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-45fcada4-e9a8-411b-851e-246be9130ccd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:19:29.672327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:29.672327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:29.672327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:29.672327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:29.672327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:29.672327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:29.672327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:29.672327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:29.672327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:29.672327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:29.672327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:29.672327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:29.672327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:29.672327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:29.672327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:29.673260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:29.673260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:29.673260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:29.673260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:29.673260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:29.673260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:29.673260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:29.673260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:29.673260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:29.673260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:29.673260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:29.673260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:29.673260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:29.673260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:29.673260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:29.673260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:29.674082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:29.674082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:29.674082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:29.674082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:29.674082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:29.674082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:29.674082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:29.674082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:29.674082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:29.674082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:29.674082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:19:29.674082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:19:29.674082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 33/68] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:19:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:29.680106 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f790c-e19d-4f4b-aa67-7f935b8c906b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:29.680106 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f790c-e19d-4f4b-aa67-7f935b8c906b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:29.680106 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f790c-e19d-4f4b-aa67-7f935b8c906b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:29.696365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-466f790c-e19d-4f4b-aa67-7f935b8c906b None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:29.697573 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-466f790c-e19d-4f4b-aa67-7f935b8c906b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:29.732952 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 36/69] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:29.742825 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0bfe3-3d7c-4e11-a11a-b17ca7dd754a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:29.770883 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 34/70] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:29.771737 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f790c-e19d-4f4b-aa67-7f935b8c906b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:29.840867 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:29.866355 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:29.869669 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0bfe3-3d7c-4e11-a11a-b17ca7dd754a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:29.891876 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f790c-e19d-4f4b-aa67-7f935b8c906b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:29.914221 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724fded-819c-4f32-b29e-2b51163b14db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:29.914888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724fded-819c-4f32-b29e-2b51163b14db None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:29.915168 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724fded-819c-4f32-b29e-2b51163b14db None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:29.915442 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724fded-819c-4f32-b29e-2b51163b14db None admin] PATH_INFO: `/v3/roles/member` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:29.926634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724fded-819c-4f32-b29e-2b51163b14db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:29.931768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-8724fded-819c-4f32-b29e-2b51163b14db None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:19:29.931768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:29.931768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:29.931768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:29.931768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:29.931768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:29.931768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:29.931768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:29.931768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:29.931768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:29.931768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:29.931768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:29.931768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:29.931768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:29.931768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:29.931768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:29.932614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:29.932614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:29.932614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:29.932614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:29.932614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:29.932614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:29.932614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:29.932614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:29.932614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:29.932614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:29.932614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:29.932614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:29.932614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:29.932614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:29.932614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:29.932614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:29.933764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:29.933764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:29.933764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:29.933764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:29.933764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:29.933764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:29.933764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:29.933764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:29.933764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:29.933764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:29.933764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:19:29.933764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:19:29.933764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 37/71] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:29.933764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724fded-819c-4f32-b29e-2b51163b14db None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:29.935380 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0fee5-6e05-487e-94da-f5e5dda7bfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:29.935380 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0fee5-6e05-487e-94da-f5e5dda7bfc0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:29.935380 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0fee5-6e05-487e-94da-f5e5dda7bfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:29.935380 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0fee5-6e05-487e-94da-f5e5dda7bfc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:29.935696 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac0fee5-6e05-487e-94da-f5e5dda7bfc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:29.936002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac0fee5-6e05-487e-94da-f5e5dda7bfc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:29.937404 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac0fee5-6e05-487e-94da-f5e5dda7bfc0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:29.947508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 35/72] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:29.950231 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0fee5-6e05-487e-94da-f5e5dda7bfc0 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:30.039059 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724fded-819c-4f32-b29e-2b51163b14db None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:30.052322 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0fee5-6e05-487e-94da-f5e5dda7bfc0 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:30.074201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:30.078075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce6481c-abcd-4c55-ab6e-3b1d850a24f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:30.078731 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6481c-abcd-4c55-ab6e-3b1d850a24f5 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:30.079012 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6481c-abcd-4c55-ab6e-3b1d850a24f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:30.079289 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6481c-abcd-4c55-ab6e-3b1d850a24f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:30.088748 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea56108-bf7f-4cf0-80ab-023ce9512e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:30.089398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea56108-bf7f-4cf0-80ab-023ce9512e2b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:30.089526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce6481c-abcd-4c55-ab6e-3b1d850a24f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:30.089688 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea56108-bf7f-4cf0-80ab-023ce9512e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:30.090127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea56108-bf7f-4cf0-80ab-023ce9512e2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:30.094621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-0ce6481c-abcd-4c55-ab6e-3b1d850a24f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:19:30.094621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:30.094621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:30.094621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:30.094621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:30.094621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:30.094621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:30.094621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:30.094621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:30.094621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:30.094621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:30.094621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:30.094621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:30.094621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:30.094621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:30.094621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:30.095618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:30.095618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:30.095618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:30.095618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:30.095618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:30.095618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:30.095618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:30.095618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:30.095618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:30.095618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:30.095618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:30.095618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:30.095618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:30.095618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:30.095618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:30.095618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:30.096417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:30.096417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:30.096417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:30.096417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:30.096417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:30.096417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:30.096417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:30.096417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:30.096417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:30.096417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:30.096417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:19:30.096417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:19:30.096417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 38/73] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:19:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:30.097258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b206fb-fcf9-4356-a514-5a469134785a None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:30.097258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b206fb-fcf9-4356-a514-5a469134785a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:30.097540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b206fb-fcf9-4356-a514-5a469134785a None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:30.098884 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 39/74] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:30.099386 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b206fb-fcf9-4356-a514-5a469134785a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:30.106919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea56108-bf7f-4cf0-80ab-023ce9512e2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:30.111974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-1ea56108-bf7f-4cf0-80ab-023ce9512e2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:19:30.111974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:30.111974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:30.111974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:30.111974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:30.111974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:30.111974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:30.111974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:30.111974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:30.111974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:30.111974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:30.111974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:30.111974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:30.111974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:30.111974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:30.111974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:30.112818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:30.112818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:30.112818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:30.112818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:30.112818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:30.112818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:30.112818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:30.112818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:30.112818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:30.112818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:30.112818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:30.112818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:30.112818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:30.112818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:30.112818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:30.112818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:30.113566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:30.113566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:30.113566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:30.113566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:30.113566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:30.113566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:30.113566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:30.113566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:30.113566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:30.113566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:30.113566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:19:30.113566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:19:30.113566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 36/75] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:30.114228 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea56108-bf7f-4cf0-80ab-023ce9512e2b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:30.127621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:30.211077 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b206fb-fcf9-4356-a514-5a469134785a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:30.218774 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea56108-bf7f-4cf0-80ab-023ce9512e2b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:30.246626 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85fbf6-41c6-4c74-a9d6-f57238865611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:30.247376 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85fbf6-41c6-4c74-a9d6-f57238865611 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:30.247660 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85fbf6-41c6-4c74-a9d6-f57238865611 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:30.247984 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85fbf6-41c6-4c74-a9d6-f57238865611 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:30.248720 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85fbf6-41c6-4c74-a9d6-f57238865611 None admin] RBAC: Adding query filter params (name=member) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:30.249098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85fbf6-41c6-4c74-a9d6-f57238865611 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:30.250620 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85fbf6-41c6-4c74-a9d6-f57238865611 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:30.254415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c536d5a-b71b-4a76-9e2d-0fbc2b9fa0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:30.255069 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c536d5a-b71b-4a76-9e2d-0fbc2b9fa0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:30.255361 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c536d5a-b71b-4a76-9e2d-0fbc2b9fa0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:30.255650 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c536d5a-b71b-4a76-9e2d-0fbc2b9fa0e9 None admin] PATH_INFO: `/v3/users/01b3cc95b90b46cf85a902fea655d1ea` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:30.256860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c536d5a-b71b-4a76-9e2d-0fbc2b9fa0e9 None admin] RBAC: Authorizing `identity:get_user(user_id=01b3cc95b90b46cf85a902fea655d1ea)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:30.258366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c536d5a-b71b-4a76-9e2d-0fbc2b9fa0e9 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:30.259303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 40/76] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:19:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:30.260241 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85fbf6-41c6-4c74-a9d6-f57238865611 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:30.267021 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 37/77] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:30 2022] GET /identity/v3/users/01b3cc95b90b46cf85a902fea655d1ea => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:30.267021 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdaf91-5440-417f-8243-c7834f535266 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:30.267021 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdaf91-5440-417f-8243-c7834f535266 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:30.267021 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdaf91-5440-417f-8243-c7834f535266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:30.283104 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f0bdaf91-5440-417f-8243-c7834f535266 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:30.284303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f0bdaf91-5440-417f-8243-c7834f535266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:30.328274 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:30.343931 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:30.355585 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 38/78] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:30.355585 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdaf91-5440-417f-8243-c7834f535266 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:30.367868 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85fbf6-41c6-4c74-a9d6-f57238865611 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:30.402596 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc79a4-f58c-4656-ba05-fd237374e168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:30.403219 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc79a4-f58c-4656-ba05-fd237374e168 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:30.403507 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc79a4-f58c-4656-ba05-fd237374e168 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:30.403773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc79a4-f58c-4656-ba05-fd237374e168 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:30.404478 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc79a4-f58c-4656-ba05-fd237374e168 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:30.404779 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc79a4-f58c-4656-ba05-fd237374e168 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:30.406187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc79a4-f58c-4656-ba05-fd237374e168 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:30.414672 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 41/79] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:19:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:30.415206 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc79a4-f58c-4656-ba05-fd237374e168 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:30.460778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdaf91-5440-417f-8243-c7834f535266 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:30.494375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda94ef-24ba-4ffe-8ea6-96d11e40b124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:30.495061 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda94ef-24ba-4ffe-8ea6-96d11e40b124 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:30.495882 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda94ef-24ba-4ffe-8ea6-96d11e40b124 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:30.496167 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda94ef-24ba-4ffe-8ea6-96d11e40b124 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:30.496989 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda94ef-24ba-4ffe-8ea6-96d11e40b124 None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:30.497296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda94ef-24ba-4ffe-8ea6-96d11e40b124 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:30.498721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda94ef-24ba-4ffe-8ea6-96d11e40b124 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:30.506892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 39/80] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:19:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:30.507707 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda94ef-24ba-4ffe-8ea6-96d11e40b124 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:30.520852 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc79a4-f58c-4656-ba05-fd237374e168 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:30.552698 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:30.555384 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dadec2-86c9-4871-bc2f-1e1918836851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:30.556030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dadec2-86c9-4871-bc2f-1e1918836851 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:30.556312 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dadec2-86c9-4871-bc2f-1e1918836851 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:30.556582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dadec2-86c9-4871-bc2f-1e1918836851 None admin] PATH_INFO: `/v3/users/8d5b327a728e436b857d5d3b3675bc2e` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:30.572742 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dadec2-86c9-4871-bc2f-1e1918836851 None admin] RBAC: Authorizing `identity:get_user(user_id=8d5b327a728e436b857d5d3b3675bc2e)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:30.574509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dadec2-86c9-4871-bc2f-1e1918836851 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:30.576364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 42/81] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:30 2022] GET /identity/v3/users/8d5b327a728e436b857d5d3b3675bc2e => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:30.577255 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dadec2-86c9-4871-bc2f-1e1918836851 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:30.579845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:30.614011 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda94ef-24ba-4ffe-8ea6-96d11e40b124 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:30.650198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c5c8b-faf8-4271-acd6-4cdf148957a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:30.650917 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c5c8b-faf8-4271-acd6-4cdf148957a7 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:30.651211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c5c8b-faf8-4271-acd6-4cdf148957a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:30.651494 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c5c8b-faf8-4271-acd6-4cdf148957a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:30.680267 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dadec2-86c9-4871-bc2f-1e1918836851 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:30.715539 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62ce09-7cb1-4584-a864-60d0e9e2b64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:30.716185 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62ce09-7cb1-4584-a864-60d0e9e2b64d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:30.716466 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62ce09-7cb1-4584-a864-60d0e9e2b64d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:30.716741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62ce09-7cb1-4584-a864-60d0e9e2b64d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:30.717900 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc62ce09-7cb1-4584-a864-60d0e9e2b64d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:30.719359 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc62ce09-7cb1-4584-a864-60d0e9e2b64d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:30.720269 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c5c8b-faf8-4271-acd6-4cdf148957a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:30.720973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 43/82] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:19:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:30.725863 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62ce09-7cb1-4584-a864-60d0e9e2b64d None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:30.726004 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c5c8b-faf8-4271-acd6-4cdf148957a7 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:30.766871 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:30.790512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:30.822724 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62ce09-7cb1-4584-a864-60d0e9e2b64d None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:30.825909 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 40/83] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:19:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:19:30.826594 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c5c8b-faf8-4271-acd6-4cdf148957a7 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:30.858953 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae2969-140a-4aa6-bb6d-a9cf32752fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:30.859618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae2969-140a-4aa6-bb6d-a9cf32752fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:30.859885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae2969-140a-4aa6-bb6d-a9cf32752fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:30.860169 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae2969-140a-4aa6-bb6d-a9cf32752fe9 None admin] PATH_INFO: `/v3/users/8d5b327a728e436b857d5d3b3675bc2e` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:30.861334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae2969-140a-4aa6-bb6d-a9cf32752fe9 None admin] RBAC: Authorizing `identity:get_user(user_id=8d5b327a728e436b857d5d3b3675bc2e)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:30.862818 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae2969-140a-4aa6-bb6d-a9cf32752fe9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:30.864747 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 44/84] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:30 2022] GET /identity/v3/users/8d5b327a728e436b857d5d3b3675bc2e => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:30.865200 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae2969-140a-4aa6-bb6d-a9cf32752fe9 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:30.933260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c5c8b-faf8-4271-acd6-4cdf148957a7 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:30.966038 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae2969-140a-4aa6-bb6d-a9cf32752fe9 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:30.967582 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53693d-714f-4159-a9fb-0bd1e0f4c120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:30.968317 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53693d-714f-4159-a9fb-0bd1e0f4c120 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:30.968549 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53693d-714f-4159-a9fb-0bd1e0f4c120 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:30.968783 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53693d-714f-4159-a9fb-0bd1e0f4c120 None admin] PATH_INFO: `/v3/users/8d5b327a728e436b857d5d3b3675bc2e` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:30.969999 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53693d-714f-4159-a9fb-0bd1e0f4c120 None admin] RBAC: Authorizing `identity:get_user(user_id=8d5b327a728e436b857d5d3b3675bc2e)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:30.971460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53693d-714f-4159-a9fb-0bd1e0f4c120 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:30.973066 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 41/85] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:30 2022] GET /identity/v3/users/8d5b327a728e436b857d5d3b3675bc2e => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:30.973813 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53693d-714f-4159-a9fb-0bd1e0f4c120 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:30.984713 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:31.005169 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ec5cb-fd32-4804-bc08-25164d2e1ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:31.005804 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ec5cb-fd32-4804-bc08-25164d2e1ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:31.006119 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ec5cb-fd32-4804-bc08-25164d2e1ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:31.006400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ec5cb-fd32-4804-bc08-25164d2e1ca9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:31.014431 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:31.076791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53693d-714f-4159-a9fb-0bd1e0f4c120 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:31.079133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ec5cb-fd32-4804-bc08-25164d2e1ca9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:31.080679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ec5cb-fd32-4804-bc08-25164d2e1ca9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:31.111883 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3d8f9-0c96-4d83-a05b-bf8bd8d469ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:31.112518 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3d8f9-0c96-4d83-a05b-bf8bd8d469ca None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:31.112806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3d8f9-0c96-4d83-a05b-bf8bd8d469ca None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:31.113088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3d8f9-0c96-4d83-a05b-bf8bd8d469ca None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:31.113777 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3d8f9-0c96-4d83-a05b-bf8bd8d469ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:31.115317 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3d8f9-0c96-4d83-a05b-bf8bd8d469ca None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:31.145968 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 42/86] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Aug 1 15:19:30 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:31.145968 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3d8f9-0c96-4d83-a05b-bf8bd8d469ca None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:31.190262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 45/87] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:19:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:19:31.191469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ec5cb-fd32-4804-bc08-25164d2e1ca9 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:31.226689 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:31.242406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:31.273688 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3d8f9-0c96-4d83-a05b-bf8bd8d469ca None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:31.321778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c427f8-ff39-432d-a439-8694d3602a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:31.322552 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c427f8-ff39-432d-a439-8694d3602a3d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:31.322795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ec5cb-fd32-4804-bc08-25164d2e1ca9 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:31.322898 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c427f8-ff39-432d-a439-8694d3602a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:31.323110 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c427f8-ff39-432d-a439-8694d3602a3d None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:31.324837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c427f8-ff39-432d-a439-8694d3602a3d None admin] RBAC: Authorizing `identity:get_project(project_id=44e0498d623545e68ea89aa6de0a8be7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:31.326405 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c427f8-ff39-432d-a439-8694d3602a3d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:31.328283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 43/88] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:31 2022] GET /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:31.328897 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c427f8-ff39-432d-a439-8694d3602a3d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:31.363675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea68b5-ca0b-4f8b-81e4-a63d67135f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:31.364312 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea68b5-ca0b-4f8b-81e4-a63d67135f02 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:31.364600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea68b5-ca0b-4f8b-81e4-a63d67135f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:31.364882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea68b5-ca0b-4f8b-81e4-a63d67135f02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:31.457539 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c427f8-ff39-432d-a439-8694d3602a3d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:31.466014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ea68b5-ca0b-4f8b-81e4-a63d67135f02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:31.467127 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ea68b5-ca0b-4f8b-81e4-a63d67135f02 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:31.495345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:31.502737 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5c4c8-dbc6-4e73-828b-9b30ede1c814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:31.503662 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5c4c8-dbc6-4e73-828b-9b30ede1c814 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:31.504019 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5c4c8-dbc6-4e73-828b-9b30ede1c814 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:31.504309 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5c4c8-dbc6-4e73-828b-9b30ede1c814 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:31.514269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:31.608208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5c4c8-dbc6-4e73-828b-9b30ede1c814 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:31.609874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5c4c8-dbc6-4e73-828b-9b30ede1c814 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:31.610609 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 46/89] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:19:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:19:31.611491 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea68b5-ca0b-4f8b-81e4-a63d67135f02 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:31.747808 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 44/90] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:19:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:19:31.749160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5c4c8-dbc6-4e73-828b-9b30ede1c814 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:31.754354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea68b5-ca0b-4f8b-81e4-a63d67135f02 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:31.785242 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:31.793872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf1adf-22cd-4f48-b869-b62a56454919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:31.794561 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf1adf-22cd-4f48-b869-b62a56454919 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:31.794993 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf1adf-22cd-4f48-b869-b62a56454919 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:31.795271 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf1adf-22cd-4f48-b869-b62a56454919 None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:31.796486 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf1adf-22cd-4f48-b869-b62a56454919 None admin] RBAC: Authorizing `identity:get_project(project_id=44e0498d623545e68ea89aa6de0a8be7)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:31.798114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf1adf-22cd-4f48-b869-b62a56454919 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:31.800236 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 47/91] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:31 2022] GET /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:31.801166 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf1adf-22cd-4f48-b869-b62a56454919 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:31.810800 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:31.875810 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5c4c8-dbc6-4e73-828b-9b30ede1c814 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:31.919835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf1adf-22cd-4f48-b869-b62a56454919 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:31.925654 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291079c1-3255-4b8c-a64a-08c3b41e2d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:31.926400 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291079c1-3255-4b8c-a64a-08c3b41e2d82 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:31.926684 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291079c1-3255-4b8c-a64a-08c3b41e2d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:31.926961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291079c1-3255-4b8c-a64a-08c3b41e2d82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:31.927756 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291079c1-3255-4b8c-a64a-08c3b41e2d82 None admin] RBAC: Adding query filter params (role.id=7bdfa43dc1d046f7965ffe52b74bc58f, scope.project.id=44e0498d623545e68ea89aa6de0a8be7, user.id=01b3cc95b90b46cf85a902fea655d1ea) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:31.928066 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291079c1-3255-4b8c-a64a-08c3b41e2d82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:31.930319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291079c1-3255-4b8c-a64a-08c3b41e2d82 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:31.938684 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 45/92] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:19:31 2022] GET /identity/v3/role_assignments?user.id=01b3cc95b90b46cf85a902fea655d1ea&scope.project.id=44e0498d623545e68ea89aa6de0a8be7&role.id=7bdfa43dc1d046f7965ffe52b74bc58f => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:31.940566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291079c1-3255-4b8c-a64a-08c3b41e2d82 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:31.987742 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af6eb3-0189-4a3c-a9db-9e51b3ce1543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:31.988773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af6eb3-0189-4a3c-a9db-9e51b3ce1543 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:31.989256 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af6eb3-0189-4a3c-a9db-9e51b3ce1543 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:31.989726 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af6eb3-0189-4a3c-a9db-9e51b3ce1543 None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:31.991199 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af6eb3-0189-4a3c-a9db-9e51b3ce1543 None admin] RBAC: Authorizing `identity:get_project(project_id=44e0498d623545e68ea89aa6de0a8be7)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:31.993040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af6eb3-0189-4a3c-a9db-9e51b3ce1543 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:32.005292 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 48/93] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:31 2022] GET /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:32.006643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af6eb3-0189-4a3c-a9db-9e51b3ce1543 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:32.118822 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:32.153330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af6eb3-0189-4a3c-a9db-9e51b3ce1543 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:32.181224 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:32.190671 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291079c1-3255-4b8c-a64a-08c3b41e2d82 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:32.208392 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c10a50-652c-4a39-a0ae-925e9c850903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:32.209902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c10a50-652c-4a39-a0ae-925e9c850903 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:32.210558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c10a50-652c-4a39-a0ae-925e9c850903 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:32.211040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c10a50-652c-4a39-a0ae-925e9c850903 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:32.212271 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c10a50-652c-4a39-a0ae-925e9c850903 None admin] RBAC: Adding query filter params (role.id=9cda4e2827dc4c2e87477bef8f319f92, scope.project.id=44e0498d623545e68ea89aa6de0a8be7, user.id=8d5b327a728e436b857d5d3b3675bc2e) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:32.213214 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c10a50-652c-4a39-a0ae-925e9c850903 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:32.214924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c10a50-652c-4a39-a0ae-925e9c850903 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:32.221843 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 49/94] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:19:32 2022] GET /identity/v3/role_assignments?user.id=8d5b327a728e436b857d5d3b3675bc2e&scope.project.id=44e0498d623545e68ea89aa6de0a8be7&role.id=9cda4e2827dc4c2e87477bef8f319f92 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:32.227444 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c10a50-652c-4a39-a0ae-925e9c850903 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:32.286812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e09bd-345f-4eb6-8c58-e9133beda274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:32.287423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e09bd-345f-4eb6-8c58-e9133beda274 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:32.287767 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e09bd-345f-4eb6-8c58-e9133beda274 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:32.288054 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e09bd-345f-4eb6-8c58-e9133beda274 None admin] PATH_INFO: `/v3/projects/7cb636795baf4f7c85f61551f54b1f4e` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:32.293501 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e09bd-345f-4eb6-8c58-e9133beda274 None admin] RBAC: Authorizing `identity:get_project(project_id=7cb636795baf4f7c85f61551f54b1f4e)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:32.295109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e09bd-345f-4eb6-8c58-e9133beda274 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:32.297263 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 46/95] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:31 2022] GET /identity/v3/projects/7cb636795baf4f7c85f61551f54b1f4e => generated 312 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:32.303184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e09bd-345f-4eb6-8c58-e9133beda274 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:32.371131 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c10a50-652c-4a39-a0ae-925e9c850903 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:32.413786 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26349d22-9da2-4464-8ea8-72331bd3804d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:32.414808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26349d22-9da2-4464-8ea8-72331bd3804d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:32.415292 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26349d22-9da2-4464-8ea8-72331bd3804d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:32.415787 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26349d22-9da2-4464-8ea8-72331bd3804d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:32.418043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26349d22-9da2-4464-8ea8-72331bd3804d None admin] RBAC: Adding query filter params (role.id=73e29e4f51e6469facf94aaa698c5cfe, scope.project.id=44e0498d623545e68ea89aa6de0a8be7, user.id=8d5b327a728e436b857d5d3b3675bc2e) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:32.418952 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26349d22-9da2-4464-8ea8-72331bd3804d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:32.420652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26349d22-9da2-4464-8ea8-72331bd3804d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:32.428884 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 50/96] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:19:32 2022] GET /identity/v3/role_assignments?user.id=8d5b327a728e436b857d5d3b3675bc2e&scope.project.id=44e0498d623545e68ea89aa6de0a8be7&role.id=73e29e4f51e6469facf94aaa698c5cfe => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:32.523342 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e09bd-345f-4eb6-8c58-e9133beda274 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:32.577633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07427389-89ad-410d-9f93-567be75f67f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:32.578221 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07427389-89ad-410d-9f93-567be75f67f6 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:32.578506 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07427389-89ad-410d-9f93-567be75f67f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:32.578781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07427389-89ad-410d-9f93-567be75f67f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:32.579600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07427389-89ad-410d-9f93-567be75f67f6 None admin] RBAC: Adding query filter params (role.id=9cda4e2827dc4c2e87477bef8f319f92, scope.project.id=7cb636795baf4f7c85f61551f54b1f4e, user.id=8d5b327a728e436b857d5d3b3675bc2e) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:32.579912 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07427389-89ad-410d-9f93-567be75f67f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:32.581405 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07427389-89ad-410d-9f93-567be75f67f6 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:32.591480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 47/97] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:19:32 2022] GET /identity/v3/role_assignments?user.id=8d5b327a728e436b857d5d3b3675bc2e&scope.project.id=7cb636795baf4f7c85f61551f54b1f4e&role.id=9cda4e2827dc4c2e87477bef8f319f92 => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:33.871895 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e038869-4538-4ca1-b8bd-b5f00dd02b78 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:33.872929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e038869-4538-4ca1-b8bd-b5f00dd02b78 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:33.873423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e038869-4538-4ca1-b8bd-b5f00dd02b78 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:33.875020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 51/98] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:33.880454 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6767995-cadf-465d-aa3a-95c7f1f6080f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:33.881250 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6767995-cadf-465d-aa3a-95c7f1f6080f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:33.881667 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6767995-cadf-465d-aa3a-95c7f1f6080f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:33.910975 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d6767995-cadf-465d-aa3a-95c7f1f6080f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:33.912458 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d6767995-cadf-465d-aa3a-95c7f1f6080f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:33.942956 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:33.988926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6bdf52-0bc8-4bb8-961c-f24479056901 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:33.989263 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6bdf52-0bc8-4bb8-961c-f24479056901 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:33.989485 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6bdf52-0bc8-4bb8-961c-f24479056901 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:33.990864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 52/99] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:19:34.003257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561563d2-99ec-4ffc-9fcd-34915c730af2 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:34.003817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561563d2-99ec-4ffc-9fcd-34915c730af2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:34.004293 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561563d2-99ec-4ffc-9fcd-34915c730af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:34.012342 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:34.012922 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 48/100] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:34.034925 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-561563d2-99ec-4ffc-9fcd-34915c730af2 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:34.036145 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-561563d2-99ec-4ffc-9fcd-34915c730af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:34.075776 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6767995-cadf-465d-aa3a-95c7f1f6080f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:34.180401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 53/101] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:19:34.221313 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561563d2-99ec-4ffc-9fcd-34915c730af2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:34.284079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6767995-cadf-465d-aa3a-95c7f1f6080f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:34.378102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4fe34-feb5-46c5-9bb3-0f5274be586c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:34.379149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4fe34-feb5-46c5-9bb3-0f5274be586c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:34.379640 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4fe34-feb5-46c5-9bb3-0f5274be586c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:34.380119 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4fe34-feb5-46c5-9bb3-0f5274be586c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:34.391562 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4fe34-feb5-46c5-9bb3-0f5274be586c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:34.397156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-2ad4fe34-feb5-46c5-9bb3-0f5274be586c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:19:34.397156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:34.397156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:34.397156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:34.397156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:34.397156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:34.397156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:34.397156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:34.397156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:34.397156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:34.397156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:34.397156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:34.397156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:34.397156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:34.397156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:34.397156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:34.398283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:34.398283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:34.398283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:34.398283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:34.398283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:34.398283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:34.398283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:34.398283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:34.398283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:34.398283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:34.398283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:34.398283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:34.398283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:34.398283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:34.398283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:34.398283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:34.399136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:34.399136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:34.399136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:34.399136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:34.399136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:34.399136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:34.399136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:34.399136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:34.399136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:34.399136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:34.399136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:19:34.399136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:19:34.401992 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 49/102] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:19:34.411417 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4fe34-feb5-46c5-9bb3-0f5274be586c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:34.427495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561563d2-99ec-4ffc-9fcd-34915c730af2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:34.442384 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:34.504686 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:34.518242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e87e0-3c25-4fb4-914a-d4f44eaaf6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:34.518947 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e87e0-3c25-4fb4-914a-d4f44eaaf6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:34.519230 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e87e0-3c25-4fb4-914a-d4f44eaaf6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:34.519502 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e87e0-3c25-4fb4-914a-d4f44eaaf6d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:34.522595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e87e0-3c25-4fb4-914a-d4f44eaaf6d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:34.524113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e87e0-3c25-4fb4-914a-d4f44eaaf6d8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:34.529990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 54/103] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:19:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:34.541688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e87e0-3c25-4fb4-914a-d4f44eaaf6d8 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:34.585547 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4fe34-feb5-46c5-9bb3-0f5274be586c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:34.653536 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1c597-6477-4ffd-84c5-8c5f20b060ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:34.654196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1c597-6477-4ffd-84c5-8c5f20b060ec None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:34.654481 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1c597-6477-4ffd-84c5-8c5f20b060ec None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:34.654820 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1c597-6477-4ffd-84c5-8c5f20b060ec None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:34.655614 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1c597-6477-4ffd-84c5-8c5f20b060ec None admin] RBAC: Adding query filter params (name=reader) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:34.655924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1c597-6477-4ffd-84c5-8c5f20b060ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:34.657427 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1c597-6477-4ffd-84c5-8c5f20b060ec None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:34.680625 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 50/104] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:19:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:34.682135 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1c597-6477-4ffd-84c5-8c5f20b060ec None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:34.729137 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e87e0-3c25-4fb4-914a-d4f44eaaf6d8 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:34.802760 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f888bdb-b6af-4d7c-9e31-4a32d0d742ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:34.803436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f888bdb-b6af-4d7c-9e31-4a32d0d742ba None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:34.803720 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f888bdb-b6af-4d7c-9e31-4a32d0d742ba None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:34.803989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f888bdb-b6af-4d7c-9e31-4a32d0d742ba None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:34.804674 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f888bdb-b6af-4d7c-9e31-4a32d0d742ba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:34.806134 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f888bdb-b6af-4d7c-9e31-4a32d0d742ba None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:34.818265 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:34.848057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1c597-6477-4ffd-84c5-8c5f20b060ec None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:34.863670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 55/105] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Aug 1 15:19:34 2022] POST /identity/v3/projects => generated 302 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:19:34.865093 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05547519-83fd-42e4-a730-7e3ce5f9b952 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:34.865384 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05547519-83fd-42e4-a730-7e3ce5f9b952 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:34.865663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05547519-83fd-42e4-a730-7e3ce5f9b952 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:34.870391 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 56/106] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:34.876462 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5d3b5-d0bb-4226-bc33-c0bb199001e0 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:34.877052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5d3b5-d0bb-4226-bc33-c0bb199001e0 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:34.877052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5d3b5-d0bb-4226-bc33-c0bb199001e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:34.916592 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c0f5d3b5-d0bb-4226-bc33-c0bb199001e0 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:34.918492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c0f5d3b5-d0bb-4226-bc33-c0bb199001e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:34.935467 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5d096-00ff-4adf-8974-d629a57f99ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:34.936150 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5d096-00ff-4adf-8974-d629a57f99ee None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:34.936435 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5d096-00ff-4adf-8974-d629a57f99ee None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:34.936721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5d096-00ff-4adf-8974-d629a57f99ee None admin] PATH_INFO: `/v3/users/77510eb658e346728640f3cbede2912a` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:34.980211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5d096-00ff-4adf-8974-d629a57f99ee None admin] RBAC: Authorizing `identity:get_user(user_id=77510eb658e346728640f3cbede2912a)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:34.981715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5d096-00ff-4adf-8974-d629a57f99ee None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:34.983836 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 51/107] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:34 2022] GET /identity/v3/users/77510eb658e346728640f3cbede2912a => generated 289 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:35.005261 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:35.012228 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5d096-00ff-4adf-8974-d629a57f99ee None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:35.030407 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 57/108] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:35.072055 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5d3b5-d0bb-4226-bc33-c0bb199001e0 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:35.210205 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5d3b5-d0bb-4226-bc33-c0bb199001e0 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:35.244001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5d096-00ff-4adf-8974-d629a57f99ee None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:35.259717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae020c36-3f10-47c3-a794-3bb3b68971d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:35.261284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae020c36-3f10-47c3-a794-3bb3b68971d4 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:35.261776 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae020c36-3f10-47c3-a794-3bb3b68971d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:35.262278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae020c36-3f10-47c3-a794-3bb3b68971d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:35.285680 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae020c36-3f10-47c3-a794-3bb3b68971d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:35.295614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-ae020c36-3f10-47c3-a794-3bb3b68971d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:19:35.295614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:35.295614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:35.295614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:35.295614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:35.295614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:35.295614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:35.295614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:35.295614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:35.295614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:35.295614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:35.295614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:35.295614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:35.295614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:35.295614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:35.295614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:35.296499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:35.296499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:35.296499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:35.296499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:35.296499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:35.296499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:35.296499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:35.296499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:35.296499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:35.296499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:35.296499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:35.296499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:35.296499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:35.296499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:35.296499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:35.296499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:35.297281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:35.297281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:35.297281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:35.297281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:35.297281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:35.297281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:35.297281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:35.297281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:35.297281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:35.297281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:35.297281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:19:35.297281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:19:35.297281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 58/109] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:19:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:19:35.298788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322900a-f9b6-4e7d-9ce8-d4e5ea4382f1 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:35.298788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322900a-f9b6-4e7d-9ce8-d4e5ea4382f1 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:35.300116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322900a-f9b6-4e7d-9ce8-d4e5ea4382f1 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:35.300668 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 59/110] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:35.305458 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322900a-f9b6-4e7d-9ce8-d4e5ea4382f1 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:35.307037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b08c2-9ece-425f-9bd1-16bb75a76e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:35.308277 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b08c2-9ece-425f-9bd1-16bb75a76e2c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:35.308565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b08c2-9ece-425f-9bd1-16bb75a76e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:35.308842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b08c2-9ece-425f-9bd1-16bb75a76e2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:35.320619 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:35.358103 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:35.406271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901b08c2-9ece-425f-9bd1-16bb75a76e2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:35.408953 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901b08c2-9ece-425f-9bd1-16bb75a76e2c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:35.425389 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322900a-f9b6-4e7d-9ce8-d4e5ea4382f1 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:35.490784 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16eb612-a519-424c-9401-751b0168a399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:35.491429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16eb612-a519-424c-9401-751b0168a399 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:35.491717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16eb612-a519-424c-9401-751b0168a399 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:35.491997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16eb612-a519-424c-9401-751b0168a399 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:35.492728 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16eb612-a519-424c-9401-751b0168a399 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:35.493276 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16eb612-a519-424c-9401-751b0168a399 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:35.494764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16eb612-a519-424c-9401-751b0168a399 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:35.519269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 60/111] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:35.520541 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ca90f-3cf3-4132-94b9-3018b6b052d2 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:35.521684 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ca90f-3cf3-4132-94b9-3018b6b052d2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:35.522000 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ca90f-3cf3-4132-94b9-3018b6b052d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:35.562982 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-450ca90f-3cf3-4132-94b9-3018b6b052d2 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:35.564558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-450ca90f-3cf3-4132-94b9-3018b6b052d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:35.587987 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 52/112] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:19:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:19:35.588887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b08c2-9ece-425f-9bd1-16bb75a76e2c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:35.638869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:35.667508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 61/113] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:35.675033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ca90f-3cf3-4132-94b9-3018b6b052d2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:35.710288 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b08c2-9ece-425f-9bd1-16bb75a76e2c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:35.733246 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:35.749560 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d8622-e019-4053-ba0f-99089541ac2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:35.750184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d8622-e019-4053-ba0f-99089541ac2e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:35.750817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d8622-e019-4053-ba0f-99089541ac2e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:35.751127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d8622-e019-4053-ba0f-99089541ac2e None admin] PATH_INFO: `/v3/users/01b3cc95b90b46cf85a902fea655d1ea` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:35.752341 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d8622-e019-4053-ba0f-99089541ac2e None admin] RBAC: Authorizing `identity:get_user(user_id=01b3cc95b90b46cf85a902fea655d1ea)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:35.754501 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d8622-e019-4053-ba0f-99089541ac2e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:35.756478 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 53/114] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:35 2022] GET /identity/v3/users/01b3cc95b90b46cf85a902fea655d1ea => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:35.758051 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f431ece-1cde-4913-b207-bac9e6d2cf45 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:35.758343 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f431ece-1cde-4913-b207-bac9e6d2cf45 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:35.758794 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f431ece-1cde-4913-b207-bac9e6d2cf45 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:35.764193 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 54/115] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:35.764193 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f431ece-1cde-4913-b207-bac9e6d2cf45 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:35.814980 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ca90f-3cf3-4132-94b9-3018b6b052d2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:35.851927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca6e8a-826f-44b2-ad63-c828c1516a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:35.852590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca6e8a-826f-44b2-ad63-c828c1516a4b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:35.852873 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca6e8a-826f-44b2-ad63-c828c1516a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:35.853150 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca6e8a-826f-44b2-ad63-c828c1516a4b None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:35.854369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca6e8a-826f-44b2-ad63-c828c1516a4b None admin] RBAC: Authorizing `identity:get_project(project_id=44e0498d623545e68ea89aa6de0a8be7)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:35.856366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca6e8a-826f-44b2-ad63-c828c1516a4b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:35.858227 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 62/116] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:35 2022] GET /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:35.859009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca6e8a-826f-44b2-ad63-c828c1516a4b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:35.888719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f431ece-1cde-4913-b207-bac9e6d2cf45 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:35.898233 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:35.929572 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dba8e8-420d-4bbc-870a-bb27ce618507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:35.930360 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dba8e8-420d-4bbc-870a-bb27ce618507 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:35.930646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dba8e8-420d-4bbc-870a-bb27ce618507 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:35.930926 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dba8e8-420d-4bbc-870a-bb27ce618507 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:35.941805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dba8e8-420d-4bbc-870a-bb27ce618507 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:35.947026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-16dba8e8-420d-4bbc-870a-bb27ce618507 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:19:35.947026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:35.947026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:35.947026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:35.947026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:35.947026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:35.947026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:35.947026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:35.947026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:35.947026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:35.947026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:35.947026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:35.947026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:35.947026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:35.947026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:35.947026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:35.947985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:35.947985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:35.947985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:35.947985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:35.947985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:35.947985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:35.947985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:35.947985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:35.947985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:35.947985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:35.947985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:35.947985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:35.947985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:35.947985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:35.947985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:35.947985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:35.948899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:35.948899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:35.948899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:35.948899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:35.948899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:35.948899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:35.948899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:35.948899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:35.948899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:35.948899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:35.948899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:19:35.948899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:19:35.948899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 55/117] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:35.949782 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b8c4e-a0b0-4c6a-a28d-d86d742a3a54 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:35.950096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b8c4e-a0b0-4c6a-a28d-d86d742a3a54 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:35.950376 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b8c4e-a0b0-4c6a-a28d-d86d742a3a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:35.969107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d23b8c4e-a0b0-4c6a-a28d-d86d742a3a54 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:35.970349 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d23b8c4e-a0b0-4c6a-a28d-d86d742a3a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:35.986927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca6e8a-826f-44b2-ad63-c828c1516a4b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:35.989469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:36.028652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6202f0-ee4b-4dbf-92ed-f33719110a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:36.029225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6202f0-ee4b-4dbf-92ed-f33719110a87 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:36.029509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6202f0-ee4b-4dbf-92ed-f33719110a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:36.029784 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6202f0-ee4b-4dbf-92ed-f33719110a87 None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:36.031030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6202f0-ee4b-4dbf-92ed-f33719110a87 None admin] RBAC: Authorizing `identity:get_project(project_id=44e0498d623545e68ea89aa6de0a8be7)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:36.032827 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6202f0-ee4b-4dbf-92ed-f33719110a87 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:36.034836 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 63/118] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:35 2022] GET /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:36.035951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12b813-ed85-4304-baa4-96764627a0c3 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:36.036243 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12b813-ed85-4304-baa4-96764627a0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:36.036619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12b813-ed85-4304-baa4-96764627a0c3 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:36.037850 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 64/119] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:36.046006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 56/120] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:36.046006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b8c4e-a0b0-4c6a-a28d-d86d742a3a54 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:36.046247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12b813-ed85-4304-baa4-96764627a0c3 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:36.145050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:36.152503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b8c4e-a0b0-4c6a-a28d-d86d742a3a54 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:36.165406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12b813-ed85-4304-baa4-96764627a0c3 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:36.191064 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb028b64-3434-4499-aaf3-c39b2724d596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:36.191831 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb028b64-3434-4499-aaf3-c39b2724d596 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:36.192178 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb028b64-3434-4499-aaf3-c39b2724d596 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:36.192452 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb028b64-3434-4499-aaf3-c39b2724d596 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:36.193170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb028b64-3434-4499-aaf3-c39b2724d596 None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:36.193644 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb028b64-3434-4499-aaf3-c39b2724d596 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:36.195140 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb028b64-3434-4499-aaf3-c39b2724d596 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:36.201869 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b8c6e-eb73-4ebc-8c07-f2d87989d9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:36.202519 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b8c6e-eb73-4ebc-8c07-f2d87989d9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:36.202834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b8c6e-eb73-4ebc-8c07-f2d87989d9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:36.203211 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b8c6e-eb73-4ebc-8c07-f2d87989d9a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:36.203803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 57/121] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:19:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:36.204463 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb028b64-3434-4499-aaf3-c39b2724d596 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:36.208934 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b8c6e-eb73-4ebc-8c07-f2d87989d9a2 None admin] RBAC: Adding query filter params (role.id=fdbe7b923fed4101826cfb93a92e3fbf, scope.project.id=44e0498d623545e68ea89aa6de0a8be7, user.id=77510eb658e346728640f3cbede2912a) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:36.209264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b8c6e-eb73-4ebc-8c07-f2d87989d9a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:36.210754 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b8c6e-eb73-4ebc-8c07-f2d87989d9a2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:36.217272 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 65/122] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:19:36 2022] GET /identity/v3/role_assignments?user.id=77510eb658e346728640f3cbede2912a&scope.project.id=44e0498d623545e68ea89aa6de0a8be7&role.id=fdbe7b923fed4101826cfb93a92e3fbf => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:36.218350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41d1c7-161b-4ac4-bad9-3c376817e227 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:36.218664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41d1c7-161b-4ac4-bad9-3c376817e227 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:36.218941 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41d1c7-161b-4ac4-bad9-3c376817e227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:36.238842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-0d41d1c7-161b-4ac4-bad9-3c376817e227 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:36.240080 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0d41d1c7-161b-4ac4-bad9-3c376817e227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:36.272492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:36.326701 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 66/123] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:36.326701 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41d1c7-161b-4ac4-bad9-3c376817e227 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:36.378171 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb028b64-3434-4499-aaf3-c39b2724d596 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:36.417776 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f3bb4-b2a2-4c80-a93d-8c213912f624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:36.418615 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f3bb4-b2a2-4c80-a93d-8c213912f624 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:36.418889 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f3bb4-b2a2-4c80-a93d-8c213912f624 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:36.419181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f3bb4-b2a2-4c80-a93d-8c213912f624 None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7/users/01b3cc95b90b46cf85a902fea655d1ea/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:36.421220 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f3bb4-b2a2-4c80-a93d-8c213912f624 None admin] RBAC: Authorizing `identity:create_grant(project_id=44e0498d623545e68ea89aa6de0a8be7, user_id=01b3cc95b90b46cf85a902fea655d1ea, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:36.423014 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f3bb4-b2a2-4c80-a93d-8c213912f624 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:36.442900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 58/124] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:19:36 2022] PUT /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7/users/01b3cc95b90b46cf85a902fea655d1ea/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:36.443955 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f3bb4-b2a2-4c80-a93d-8c213912f624 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:36.478127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:36.498087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41d1c7-161b-4ac4-bad9-3c376817e227 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:36.577533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001311e-6e24-490a-9071-50cd41d7a02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:36.579556 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001311e-6e24-490a-9071-50cd41d7a02d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:36.581413 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001311e-6e24-490a-9071-50cd41d7a02d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:36.582198 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001311e-6e24-490a-9071-50cd41d7a02d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:36.610798 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001311e-6e24-490a-9071-50cd41d7a02d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:36.618860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-8001311e-6e24-490a-9071-50cd41d7a02d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:19:36.618860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:36.618860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:36.618860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:36.618860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:36.618860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:36.618860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:36.618860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:36.618860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:36.618860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:36.618860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:36.618860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:36.618860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:36.618860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:36.618860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:36.618860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:36.619788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:36.619788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:36.619788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:36.619788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:36.619788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:36.619788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:36.619788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:36.619788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:36.619788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:36.619788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:36.619788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:36.619788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:36.619788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:36.619788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:36.619788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:36.619788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:36.620600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:36.620600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:36.620600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:36.620600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:36.620600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:36.620600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:36.620600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:36.620600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:36.620600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:36.620600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:36.620600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:19:36.620600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:19:36.626806 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 67/125] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:36.627970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001311e-6e24-490a-9071-50cd41d7a02d None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:36.654260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f3bb4-b2a2-4c80-a93d-8c213912f624 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:36.662344 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:36.693364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951580e-2762-4830-8d03-7e53cea31313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:36.694670 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951580e-2762-4830-8d03-7e53cea31313 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:36.695156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951580e-2762-4830-8d03-7e53cea31313 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:36.695628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951580e-2762-4830-8d03-7e53cea31313 None admin] PATH_INFO: `/v3/users/8d5b327a728e436b857d5d3b3675bc2e` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:36.697042 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951580e-2762-4830-8d03-7e53cea31313 None admin] RBAC: Authorizing `identity:get_user(user_id=8d5b327a728e436b857d5d3b3675bc2e)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:36.699260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951580e-2762-4830-8d03-7e53cea31313 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:36.701100 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 59/126] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:36 2022] GET /identity/v3/users/8d5b327a728e436b857d5d3b3675bc2e => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:36.702089 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951580e-2762-4830-8d03-7e53cea31313 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:36.753455 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001311e-6e24-490a-9071-50cd41d7a02d None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:36.793656 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a1d3b-0a07-4232-b901-d12640da4e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:36.794456 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a1d3b-0a07-4232-b901-d12640da4e03 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:36.794747 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a1d3b-0a07-4232-b901-d12640da4e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:36.795026 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a1d3b-0a07-4232-b901-d12640da4e03 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:36.801034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:36.808062 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a1d3b-0a07-4232-b901-d12640da4e03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:36.813434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-fc0a1d3b-0a07-4232-b901-d12640da4e03 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:19:36.813434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:36.813434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:36.813434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:36.813434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:36.813434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:36.813434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:36.813434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:36.813434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:36.813434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:36.813434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:36.813434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:36.813434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:36.813434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:36.813434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:36.813434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:36.814369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:36.814369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:36.814369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:36.814369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:36.814369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:36.814369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:36.814369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:36.814369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:36.814369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:36.814369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:36.814369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:36.814369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:36.814369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:36.814369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:36.814369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:36.814369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:36.815125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:36.815125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:36.815125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:36.815125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:36.815125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:36.815125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:36.815125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:36.815125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:36.815125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:36.815125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:36.815125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:19:36.815125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:19:36.815125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 68/127] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:19:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:36.816375 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a1d3b-0a07-4232-b901-d12640da4e03 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:36.826455 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951580e-2762-4830-8d03-7e53cea31313 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:36.863083 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc780fe9-cbfb-4dbf-b85b-44dd01492598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:36.863944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc780fe9-cbfb-4dbf-b85b-44dd01492598 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:36.864369 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc780fe9-cbfb-4dbf-b85b-44dd01492598 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:36.864782 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc780fe9-cbfb-4dbf-b85b-44dd01492598 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:36.865633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc780fe9-cbfb-4dbf-b85b-44dd01492598 None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:36.866099 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc780fe9-cbfb-4dbf-b85b-44dd01492598 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:36.867692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc780fe9-cbfb-4dbf-b85b-44dd01492598 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:36.877316 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 60/128] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:19:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:36.878474 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc780fe9-cbfb-4dbf-b85b-44dd01492598 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:36.962284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:36.982761 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a1d3b-0a07-4232-b901-d12640da4e03 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:36.989005 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc780fe9-cbfb-4dbf-b85b-44dd01492598 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:37.044390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14c39c-b41c-42d4-8655-977763a5424a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:37.045054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14c39c-b41c-42d4-8655-977763a5424a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.045326 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14c39c-b41c-42d4-8655-977763a5424a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.045604 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14c39c-b41c-42d4-8655-977763a5424a None admin] PATH_INFO: `/v3/projects/7cb636795baf4f7c85f61551f54b1f4e` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.046817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b40144d-ec40-4670-8ec8-5eb892662d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:37.047446 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40144d-ec40-4670-8ec8-5eb892662d72 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.047798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40144d-ec40-4670-8ec8-5eb892662d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.048509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40144d-ec40-4670-8ec8-5eb892662d72 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.049361 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b40144d-ec40-4670-8ec8-5eb892662d72 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:37.049715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b40144d-ec40-4670-8ec8-5eb892662d72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:37.051302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b40144d-ec40-4670-8ec8-5eb892662d72 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:37.053478 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14c39c-b41c-42d4-8655-977763a5424a None admin] RBAC: Authorizing `identity:get_project(project_id=7cb636795baf4f7c85f61551f54b1f4e)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:37.054933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14c39c-b41c-42d4-8655-977763a5424a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:37.056724 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 69/129] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:36 2022] GET /identity/v3/projects/7cb636795baf4f7c85f61551f54b1f4e => generated 312 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:37.062668 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14c39c-b41c-42d4-8655-977763a5424a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:37.070313 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 61/130] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:19:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:37.071000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b40144d-ec40-4670-8ec8-5eb892662d72 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:37.101591 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:37.220828 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b40144d-ec40-4670-8ec8-5eb892662d72 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:37.257294 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255850b5-2586-4d77-b945-2b9cec2811d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:37.258019 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255850b5-2586-4d77-b945-2b9cec2811d7 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.258305 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255850b5-2586-4d77-b945-2b9cec2811d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.258585 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255850b5-2586-4d77-b945-2b9cec2811d7 None admin] PATH_INFO: `/v3/projects/7cb636795baf4f7c85f61551f54b1f4e/users/8d5b327a728e436b857d5d3b3675bc2e/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.261082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255850b5-2586-4d77-b945-2b9cec2811d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=7cb636795baf4f7c85f61551f54b1f4e, user_id=8d5b327a728e436b857d5d3b3675bc2e, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:37.262596 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255850b5-2586-4d77-b945-2b9cec2811d7 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:37.279849 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 62/131] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:19:37 2022] PUT /identity/v3/projects/7cb636795baf4f7c85f61551f54b1f4e/users/8d5b327a728e436b857d5d3b3675bc2e/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:37.280786 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255850b5-2586-4d77-b945-2b9cec2811d7 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:37.339365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14c39c-b41c-42d4-8655-977763a5424a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:37.411565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:37.413256 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b461b-e2a7-4b97-b424-90254d14673a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:37.414341 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b461b-e2a7-4b97-b424-90254d14673a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.414843 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b461b-e2a7-4b97-b424-90254d14673a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.415406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b461b-e2a7-4b97-b424-90254d14673a None admin] PATH_INFO: `/v3/users/8d5b327a728e436b857d5d3b3675bc2e` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.417717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391b461b-e2a7-4b97-b424-90254d14673a None admin] RBAC: Authorizing `identity:get_user(user_id=8d5b327a728e436b857d5d3b3675bc2e)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:37.421251 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391b461b-e2a7-4b97-b424-90254d14673a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:37.424551 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 70/132] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:37 2022] GET /identity/v3/users/8d5b327a728e436b857d5d3b3675bc2e => generated 275 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:37.430905 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b461b-e2a7-4b97-b424-90254d14673a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:37.463373 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:37.468916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255850b5-2586-4d77-b945-2b9cec2811d7 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:37.508593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdd8a7-0ba0-4e18-a851-204c09f80444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:37.509618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdd8a7-0ba0-4e18-a851-204c09f80444 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.510072 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdd8a7-0ba0-4e18-a851-204c09f80444 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.510523 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdd8a7-0ba0-4e18-a851-204c09f80444 None admin] PATH_INFO: `/v3/users/8d5b327a728e436b857d5d3b3675bc2e` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.512660 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bdd8a7-0ba0-4e18-a851-204c09f80444 None admin] RBAC: Authorizing `identity:get_user(user_id=8d5b327a728e436b857d5d3b3675bc2e)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:37.514600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bdd8a7-0ba0-4e18-a851-204c09f80444 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:37.516557 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 63/133] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:37 2022] GET /identity/v3/users/8d5b327a728e436b857d5d3b3675bc2e => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:37.522468 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40444fc-0b23-4394-a28d-ce258715102f None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.523178 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40444fc-0b23-4394-a28d-ce258715102f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.523777 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40444fc-0b23-4394-a28d-ce258715102f None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.526045 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 64/134] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:37.531133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40444fc-0b23-4394-a28d-ce258715102f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:37.575628 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b461b-e2a7-4b97-b424-90254d14673a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:37.622172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec360e-8784-4fd4-a2bc-60e83c187050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:37.622860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec360e-8784-4fd4-a2bc-60e83c187050 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.623149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec360e-8784-4fd4-a2bc-60e83c187050 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.623428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec360e-8784-4fd4-a2bc-60e83c187050 None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.625152 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec360e-8784-4fd4-a2bc-60e83c187050 None admin] RBAC: Authorizing `identity:get_project(project_id=44e0498d623545e68ea89aa6de0a8be7)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:37.626643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec360e-8784-4fd4-a2bc-60e83c187050 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:37.629053 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 71/135] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:37 2022] GET /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:37.633267 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aefccb-ab39-46e4-8025-352827c2a0c7 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.633267 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aefccb-ab39-46e4-8025-352827c2a0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.633267 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aefccb-ab39-46e4-8025-352827c2a0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.654487 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-77aefccb-ab39-46e4-8025-352827c2a0c7 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:37.656548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-77aefccb-ab39-46e4-8025-352827c2a0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:37.663456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40444fc-0b23-4394-a28d-ce258715102f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:37.704074 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca1cbb-3ad9-4d8b-a7e4-12ba4b2e92fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:37.704771 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca1cbb-3ad9-4d8b-a7e4-12ba4b2e92fb None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.705037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca1cbb-3ad9-4d8b-a7e4-12ba4b2e92fb None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.705333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca1cbb-3ad9-4d8b-a7e4-12ba4b2e92fb None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.706973 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca1cbb-3ad9-4d8b-a7e4-12ba4b2e92fb None admin] RBAC: Authorizing `identity:get_project(project_id=44e0498d623545e68ea89aa6de0a8be7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:37.708488 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca1cbb-3ad9-4d8b-a7e4-12ba4b2e92fb None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:37.710568 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 65/136] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:37 2022] GET /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:37.710971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca1cbb-3ad9-4d8b-a7e4-12ba4b2e92fb None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:37.725835 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:37.766247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 72/137] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:37.768418 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aefccb-ab39-46e4-8025-352827c2a0c7 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:37.811959 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:37.848798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca1cbb-3ad9-4d8b-a7e4-12ba4b2e92fb None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:37.898911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcac469-5303-469b-a1e3-4a2d998dba0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:37.899777 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcac469-5303-469b-a1e3-4a2d998dba0c None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.900071 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcac469-5303-469b-a1e3-4a2d998dba0c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.900642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcac469-5303-469b-a1e3-4a2d998dba0c None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7/users/8d5b327a728e436b857d5d3b3675bc2e/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.903076 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcac469-5303-469b-a1e3-4a2d998dba0c None admin] RBAC: Authorizing `identity:create_grant(project_id=44e0498d623545e68ea89aa6de0a8be7, user_id=8d5b327a728e436b857d5d3b3675bc2e, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:37.904786 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcac469-5303-469b-a1e3-4a2d998dba0c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:37.923243 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 66/138] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:19:37 2022] PUT /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7/users/8d5b327a728e436b857d5d3b3675bc2e/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:37.924461 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcac469-5303-469b-a1e3-4a2d998dba0c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:37.942973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aefccb-ab39-46e4-8025-352827c2a0c7 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:38.003350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6e1ed-3a9c-4821-ad04-dc2d5fd6c014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:38.006324 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6e1ed-3a9c-4821-ad04-dc2d5fd6c014 None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:38.007541 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6e1ed-3a9c-4821-ad04-dc2d5fd6c014 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:38.008416 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6e1ed-3a9c-4821-ad04-dc2d5fd6c014 None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7/users/8d5b327a728e436b857d5d3b3675bc2e/roles/73e29e4f51e6469facf94aaa698c5cfe` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:38.010132 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:38.012639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d6e1ed-3a9c-4821-ad04-dc2d5fd6c014 None admin] RBAC: Authorizing `identity:create_grant(project_id=44e0498d623545e68ea89aa6de0a8be7, user_id=8d5b327a728e436b857d5d3b3675bc2e, role_id=73e29e4f51e6469facf94aaa698c5cfe)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:38.015302 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d6e1ed-3a9c-4821-ad04-dc2d5fd6c014 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:38.043301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 73/139] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:19:37 2022] PUT /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7/users/8d5b327a728e436b857d5d3b3675bc2e/roles/73e29e4f51e6469facf94aaa698c5cfe => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:38.108020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcac469-5303-469b-a1e3-4a2d998dba0c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:38.174884 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b034c339-354b-4d5d-ab11-4c08e4af9f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:38.176003 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034c339-354b-4d5d-ab11-4c08e4af9f4d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:38.176500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034c339-354b-4d5d-ab11-4c08e4af9f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:38.176984 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034c339-354b-4d5d-ab11-4c08e4af9f4d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:38.179214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b034c339-354b-4d5d-ab11-4c08e4af9f4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:38.181913 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b034c339-354b-4d5d-ab11-4c08e4af9f4d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:38.185500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 67/140] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:19:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:38.191857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6e1ed-3a9c-4821-ad04-dc2d5fd6c014 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:38.364013 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:38.412987 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6e1ed-3a9c-4821-ad04-dc2d5fd6c014 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:38.488045 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1509f-895d-4499-b095-78e8f46577d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:38.489083 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1509f-895d-4499-b095-78e8f46577d3 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:38.489578 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1509f-895d-4499-b095-78e8f46577d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:38.490115 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1509f-895d-4499-b095-78e8f46577d3 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:38.491314 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1509f-895d-4499-b095-78e8f46577d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:38.494057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1509f-895d-4499-b095-78e8f46577d3 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:38.542539 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 74/141] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Aug 1 15:19:38 2022] POST /identity/v3/users => generated 283 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:19:38.717270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e5853-5dec-4b54-917e-2946f42d4908 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:38.717661 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e5853-5dec-4b54-917e-2946f42d4908 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:38.717856 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e5853-5dec-4b54-917e-2946f42d4908 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:38.719487 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 68/142] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:38.725336 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc2a3e-5ff6-4229-a11b-9564d341f3cc None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:38.725626 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc2a3e-5ff6-4229-a11b-9564d341f3cc None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:38.725920 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc2a3e-5ff6-4229-a11b-9564d341f3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:38.745785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e7fc2a3e-5ff6-4229-a11b-9564d341f3cc None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:38.747085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e7fc2a3e-5ff6-4229-a11b-9564d341f3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:38.828289 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 75/143] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:38.869282 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49e5853-5dec-4b54-917e-2946f42d4908 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:38.989158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49e5853-5dec-4b54-917e-2946f42d4908 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:39.028715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b850da5-d7a9-4bef-96ce-455421a2b4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:39.029471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b850da5-d7a9-4bef-96ce-455421a2b4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:39.029675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b850da5-d7a9-4bef-96ce-455421a2b4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:39.030006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b850da5-d7a9-4bef-96ce-455421a2b4e0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:39.038236 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:39.049273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b850da5-d7a9-4bef-96ce-455421a2b4e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:39.056275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-9b850da5-d7a9-4bef-96ce-455421a2b4e0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:19:39.056275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:39.056275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:39.056275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:39.056275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:39.056275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:39.056275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:39.056275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:39.056275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:39.056275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:39.056275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:39.056275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:39.056275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:39.056275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:39.056275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:39.056275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:39.057149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:39.057149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:39.057149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:39.057149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:39.057149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:39.057149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:39.057149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:39.057149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:39.057149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:39.057149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:39.057149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:39.057149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:39.057149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:39.057149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:39.057149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:39.057149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:39.058538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:39.058538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:39.058538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:39.058538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:39.058538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:39.058538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:39.058538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:39.058538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:39.058538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:39.058538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:39.058538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:19:39.058538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:19:39.062316 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 69/144] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:39.064879 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc2a3e-5ff6-4229-a11b-9564d341f3cc None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:39.170518 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:39.223745 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc2a3e-5ff6-4229-a11b-9564d341f3cc None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:39.270600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90905bd3-bbb2-4e10-83c5-a2d5c3c527e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:39.271577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90905bd3-bbb2-4e10-83c5-a2d5c3c527e3 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:39.272071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90905bd3-bbb2-4e10-83c5-a2d5c3c527e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:39.272639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90905bd3-bbb2-4e10-83c5-a2d5c3c527e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:39.273572 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90905bd3-bbb2-4e10-83c5-a2d5c3c527e3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:39.274204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90905bd3-bbb2-4e10-83c5-a2d5c3c527e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:39.275932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90905bd3-bbb2-4e10-83c5-a2d5c3c527e3 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:39.287198 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 76/145] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:19:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:39.293155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b850da5-d7a9-4bef-96ce-455421a2b4e0 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:39.440660 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b850da5-d7a9-4bef-96ce-455421a2b4e0 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:39.485558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284fa0a1-a0c3-4cab-a4fd-7809bc639cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:39.486251 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fa0a1-a0c3-4cab-a4fd-7809bc639cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:39.486532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fa0a1-a0c3-4cab-a4fd-7809bc639cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:39.486869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fa0a1-a0c3-4cab-a4fd-7809bc639cf3 None admin] PATH_INFO: `/v3/users/77510eb658e346728640f3cbede2912a` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:39.488127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284fa0a1-a0c3-4cab-a4fd-7809bc639cf3 None admin] RBAC: Authorizing `identity:get_user(user_id=77510eb658e346728640f3cbede2912a)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:39.489619 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284fa0a1-a0c3-4cab-a4fd-7809bc639cf3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:39.491674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 70/146] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:39 2022] GET /identity/v3/users/77510eb658e346728640f3cbede2912a => generated 289 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:39.497500 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90905bd3-bbb2-4e10-83c5-a2d5c3c527e3 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:39.630510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceac772-ac8b-444e-8d3b-1d379d41ba12 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:39.630946 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceac772-ac8b-444e-8d3b-1d379d41ba12 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:39.631203 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceac772-ac8b-444e-8d3b-1d379d41ba12 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:39.632605 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 71/147] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:39.644477 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad7687-61ba-4783-9574-3250370f6b81 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:39.644805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad7687-61ba-4783-9574-3250370f6b81 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:39.645033 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad7687-61ba-4783-9574-3250370f6b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:39.677761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-39ad7687-61ba-4783-9574-3250370f6b81 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:39.678997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-39ad7687-61ba-4783-9574-3250370f6b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:39.681139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90905bd3-bbb2-4e10-83c5-a2d5c3c527e3 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:39.730472 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:39.750467 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66129f35-dd91-4d29-8db2-bdd6cff9e28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:39.751373 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66129f35-dd91-4d29-8db2-bdd6cff9e28b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:39.751656 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66129f35-dd91-4d29-8db2-bdd6cff9e28b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:39.751933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66129f35-dd91-4d29-8db2-bdd6cff9e28b None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:39.756713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66129f35-dd91-4d29-8db2-bdd6cff9e28b None admin] RBAC: Authorizing `identity:get_project(project_id=44e0498d623545e68ea89aa6de0a8be7)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:39.758386 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66129f35-dd91-4d29-8db2-bdd6cff9e28b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:39.761244 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 77/148] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:39 2022] GET /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:39.764694 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:39.780443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 72/149] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:19:39.782260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66129f35-dd91-4d29-8db2-bdd6cff9e28b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:39.830804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad7687-61ba-4783-9574-3250370f6b81 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:39.944610 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66129f35-dd91-4d29-8db2-bdd6cff9e28b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:40.004612 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825e805-d37e-46f6-ac1b-958afe23ac0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:40.005301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825e805-d37e-46f6-ac1b-958afe23ac0e None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:40.005689 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825e805-d37e-46f6-ac1b-958afe23ac0e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:40.005998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825e805-d37e-46f6-ac1b-958afe23ac0e None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7/users/77510eb658e346728640f3cbede2912a/roles/fdbe7b923fed4101826cfb93a92e3fbf` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:40.009952 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad7687-61ba-4783-9574-3250370f6b81 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:40.014270 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825e805-d37e-46f6-ac1b-958afe23ac0e None admin] RBAC: Authorizing `identity:create_grant(project_id=44e0498d623545e68ea89aa6de0a8be7, user_id=77510eb658e346728640f3cbede2912a, role_id=fdbe7b923fed4101826cfb93a92e3fbf)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:40.016670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825e805-d37e-46f6-ac1b-958afe23ac0e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:40.042234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 78/150] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:19:39 2022] PUT /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7/users/77510eb658e346728640f3cbede2912a/roles/fdbe7b923fed4101826cfb93a92e3fbf => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:40.091459 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b0741-6ad7-49e8-abb8-b371dfb18254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:40.092832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b0741-6ad7-49e8-abb8-b371dfb18254 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:40.093315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b0741-6ad7-49e8-abb8-b371dfb18254 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:40.093814 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b0741-6ad7-49e8-abb8-b371dfb18254 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:40.113308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b0741-6ad7-49e8-abb8-b371dfb18254 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:40.119450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-e97b0741-6ad7-49e8-abb8-b371dfb18254 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:19:40.119450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:40.119450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:40.119450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:40.119450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:40.119450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:40.119450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:40.119450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:40.119450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:40.119450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:40.119450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:40.119450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:40.119450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:40.119450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:40.119450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:40.119450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:40.120951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:40.120951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:40.120951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:40.120951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:40.120951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:40.120951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:40.120951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:40.120951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:40.120951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:40.120951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:40.120951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:40.120951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:40.120951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:40.120951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:40.120951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:40.120951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:40.121741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:40.121741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:40.121741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:40.121741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:40.121741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:40.121741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:40.121741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:40.121741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:40.121741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:40.121741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:40.121741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:19:40.121741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:19:40.124613 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 73/151] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:19:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:40.130917 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825e805-d37e-46f6-ac1b-958afe23ac0e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:40.222606 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:40.287073 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825e805-d37e-46f6-ac1b-958afe23ac0e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:40.333925 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5869c8d-31e0-4874-a229-dd4369c8ee22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:40.334592 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5869c8d-31e0-4874-a229-dd4369c8ee22 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:40.334932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5869c8d-31e0-4874-a229-dd4369c8ee22 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:40.335218 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5869c8d-31e0-4874-a229-dd4369c8ee22 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:40.335967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5869c8d-31e0-4874-a229-dd4369c8ee22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:40.336277 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5869c8d-31e0-4874-a229-dd4369c8ee22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:40.337737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5869c8d-31e0-4874-a229-dd4369c8ee22 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:40.351431 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 79/152] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:40.358415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b0741-6ad7-49e8-abb8-b371dfb18254 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:40.440822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:40.499836 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b0741-6ad7-49e8-abb8-b371dfb18254 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:40.537094 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c30ad1-1b3e-49d7-86ed-5bb088a763b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:40.537756 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c30ad1-1b3e-49d7-86ed-5bb088a763b4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:40.538077 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c30ad1-1b3e-49d7-86ed-5bb088a763b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:40.538358 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c30ad1-1b3e-49d7-86ed-5bb088a763b4 None admin] PATH_INFO: `/v3/users/01b3cc95b90b46cf85a902fea655d1ea` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:40.539590 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c30ad1-1b3e-49d7-86ed-5bb088a763b4 None admin] RBAC: Authorizing `identity:get_user(user_id=01b3cc95b90b46cf85a902fea655d1ea)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:40.541073 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c30ad1-1b3e-49d7-86ed-5bb088a763b4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:40.543446 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 74/153] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:40 2022] GET /identity/v3/users/01b3cc95b90b46cf85a902fea655d1ea => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:40.549261 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5869c8d-31e0-4874-a229-dd4369c8ee22 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:40.681108 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5869c8d-31e0-4874-a229-dd4369c8ee22 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:40.718644 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:40.745931 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2107a1-d24e-4e04-8cc1-6e9649b04e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:40.746891 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2107a1-d24e-4e04-8cc1-6e9649b04e6c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:40.747361 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2107a1-d24e-4e04-8cc1-6e9649b04e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:40.747708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2107a1-d24e-4e04-8cc1-6e9649b04e6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:40.840474 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2107a1-d24e-4e04-8cc1-6e9649b04e6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:40.841898 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2107a1-d24e-4e04-8cc1-6e9649b04e6c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:40.997540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 80/154] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:19:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 15:19:41.004170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c30ad1-1b3e-49d7-86ed-5bb088a763b4 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:41.138452 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c30ad1-1b3e-49d7-86ed-5bb088a763b4 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:41.167471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:41.190476 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a1cc8-2247-475d-ae66-19aa8da55c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:41.191367 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a1cc8-2247-475d-ae66-19aa8da55c66 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:41.191745 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a1cc8-2247-475d-ae66-19aa8da55c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:41.192138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a1cc8-2247-475d-ae66-19aa8da55c66 None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:41.193972 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19a1cc8-2247-475d-ae66-19aa8da55c66 None admin] RBAC: Authorizing `identity:get_project(project_id=44e0498d623545e68ea89aa6de0a8be7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:41.196114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19a1cc8-2247-475d-ae66-19aa8da55c66 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:41.199370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 75/155] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:41 2022] GET /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:41.206146 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2107a1-d24e-4e04-8cc1-6e9649b04e6c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:41.225478 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:41.310437 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c5182-3120-4c09-b250-85581d430004 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:41.310796 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c5182-3120-4c09-b250-85581d430004 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:41.310987 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c5182-3120-4c09-b250-85581d430004 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:41.312356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 76/156] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:41.327911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7758de5-f589-426a-8bab-654d5ae3881a None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:41.328287 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7758de5-f589-426a-8bab-654d5ae3881a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:41.328485 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7758de5-f589-426a-8bab-654d5ae3881a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:41.360668 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d7758de5-f589-426a-8bab-654d5ae3881a None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:41.361876 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d7758de5-f589-426a-8bab-654d5ae3881a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:41.369066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2107a1-d24e-4e04-8cc1-6e9649b04e6c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:41.453393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828b6e5-0c00-494f-9e9b-bbafbe09125d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:41.454056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828b6e5-0c00-494f-9e9b-bbafbe09125d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:41.454399 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828b6e5-0c00-494f-9e9b-bbafbe09125d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:41.454924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828b6e5-0c00-494f-9e9b-bbafbe09125d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:41.455824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828b6e5-0c00-494f-9e9b-bbafbe09125d None admin] RBAC: Adding query filter params (role.id=7bdfa43dc1d046f7965ffe52b74bc58f, scope.project.id=44e0498d623545e68ea89aa6de0a8be7, user.id=01b3cc95b90b46cf85a902fea655d1ea) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:41.456216 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828b6e5-0c00-494f-9e9b-bbafbe09125d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:41.463943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828b6e5-0c00-494f-9e9b-bbafbe09125d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:41.477741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 81/157] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:19:41 2022] GET /identity/v3/role_assignments?user.id=01b3cc95b90b46cf85a902fea655d1ea&scope.project.id=44e0498d623545e68ea89aa6de0a8be7&role.id=7bdfa43dc1d046f7965ffe52b74bc58f => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:41.517919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 77/158] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:41.554054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828b6e5-0c00-494f-9e9b-bbafbe09125d None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:41.678474 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:41.700032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828b6e5-0c00-494f-9e9b-bbafbe09125d None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:41.744264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08162195-f1f6-4a80-a80d-ad23748bd2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:41.745786 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08162195-f1f6-4a80-a80d-ad23748bd2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:41.746297 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08162195-f1f6-4a80-a80d-ad23748bd2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:41.746784 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08162195-f1f6-4a80-a80d-ad23748bd2e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:41.758671 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08162195-f1f6-4a80-a80d-ad23748bd2e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:41.764737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-08162195-f1f6-4a80-a80d-ad23748bd2e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:19:41.764737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:41.764737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:41.764737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:41.764737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:41.764737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:41.764737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:41.764737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:41.764737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:41.764737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:41.764737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:41.764737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:41.764737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:41.764737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:41.764737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:41.764737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:41.765737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:41.765737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:41.765737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:41.765737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:41.765737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:41.765737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:41.765737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:41.765737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:41.765737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:41.765737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:41.765737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:41.765737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:41.765737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:41.765737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:41.765737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:41.765737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:41.766639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:41.766639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:41.766639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:41.766639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:41.766639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:41.766639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:41.766639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:41.766639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:41.766639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:41.766639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:41.766639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:19:41.766639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:19:41.769430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 82/159] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:41.775878 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7758de5-f589-426a-8bab-654d5ae3881a None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:41.895768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7758de5-f589-426a-8bab-654d5ae3881a None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:41.919536 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:41.943069 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a664e1d-760b-4b04-933b-b72e67d685ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:41.944089 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a664e1d-760b-4b04-933b-b72e67d685ce None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:41.944851 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a664e1d-760b-4b04-933b-b72e67d685ce None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:41.945430 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a664e1d-760b-4b04-933b-b72e67d685ce None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:41.946422 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a664e1d-760b-4b04-933b-b72e67d685ce None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:41.946972 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a664e1d-760b-4b04-933b-b72e67d685ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:41.948773 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a664e1d-760b-4b04-933b-b72e67d685ce None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:41.961416 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 78/160] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:19:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:41.971095 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08162195-f1f6-4a80-a80d-ad23748bd2e9 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:42.107387 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08162195-f1f6-4a80-a80d-ad23748bd2e9 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:42.148105 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aeb8eb-4370-4ae4-8b6c-7105b4655a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:42.149132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aeb8eb-4370-4ae4-8b6c-7105b4655a92 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:42.149693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aeb8eb-4370-4ae4-8b6c-7105b4655a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:42.150231 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aeb8eb-4370-4ae4-8b6c-7105b4655a92 None admin] PATH_INFO: `/v3/users/8d5b327a728e436b857d5d3b3675bc2e` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:42.152232 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aeb8eb-4370-4ae4-8b6c-7105b4655a92 None admin] RBAC: Authorizing `identity:get_user(user_id=8d5b327a728e436b857d5d3b3675bc2e)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:42.154072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aeb8eb-4370-4ae4-8b6c-7105b4655a92 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:42.156613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 83/161] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:41 2022] GET /identity/v3/users/8d5b327a728e436b857d5d3b3675bc2e => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:42.162318 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a664e1d-760b-4b04-933b-b72e67d685ce None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:42.296230 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a664e1d-760b-4b04-933b-b72e67d685ce None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:42.334596 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d679f-6bf6-4258-bf03-2148239f0cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:42.335652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d679f-6bf6-4258-bf03-2148239f0cff None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:42.336138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d679f-6bf6-4258-bf03-2148239f0cff None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:42.336635 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d679f-6bf6-4258-bf03-2148239f0cff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:42.394548 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:42.420574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d679f-6bf6-4258-bf03-2148239f0cff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:42.422603 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d679f-6bf6-4258-bf03-2148239f0cff None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:42.541035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 79/162] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:19:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:19:42.546593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aeb8eb-4370-4ae4-8b6c-7105b4655a92 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:42.561474 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:42.636403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9ae24-5715-4bcd-bc91-6bcf17137a83 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:42.637063 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9ae24-5715-4bcd-bc91-6bcf17137a83 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:42.637540 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9ae24-5715-4bcd-bc91-6bcf17137a83 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:42.639275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 80/163] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:42.644925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1f609-4acb-4d75-bf2c-39718f1df245 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:42.645496 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1f609-4acb-4d75-bf2c-39718f1df245 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:42.646001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1f609-4acb-4d75-bf2c-39718f1df245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:42.680835 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-02a1f609-4acb-4d75-bf2c-39718f1df245 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:42.683344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-02a1f609-4acb-4d75-bf2c-39718f1df245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:42.736853 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aeb8eb-4370-4ae4-8b6c-7105b4655a92 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:42.827315 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fabaf3-8a72-4a69-bf72-0a15d2222e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:42.828493 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fabaf3-8a72-4a69-bf72-0a15d2222e22 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:42.829869 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fabaf3-8a72-4a69-bf72-0a15d2222e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:42.831359 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fabaf3-8a72-4a69-bf72-0a15d2222e22 None admin] PATH_INFO: `/v3/projects/7cb636795baf4f7c85f61551f54b1f4e` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:42.834121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fabaf3-8a72-4a69-bf72-0a15d2222e22 None admin] RBAC: Authorizing `identity:get_project(project_id=7cb636795baf4f7c85f61551f54b1f4e)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:42.835999 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fabaf3-8a72-4a69-bf72-0a15d2222e22 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:42.838807 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 84/164] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:42 2022] GET /identity/v3/projects/7cb636795baf4f7c85f61551f54b1f4e => generated 312 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:42.841000 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67519d4e-ffd8-48a2-9913-8530c783bfc4 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:42.843030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67519d4e-ffd8-48a2-9913-8530c783bfc4 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:42.844367 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67519d4e-ffd8-48a2-9913-8530c783bfc4 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:42.846739 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 85/165] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:42.848941 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67519d4e-ffd8-48a2-9913-8530c783bfc4 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:42.852242 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 81/166] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:19:42.854946 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df9422-32eb-4cde-85a4-638cb2af6790 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:42.855466 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df9422-32eb-4cde-85a4-638cb2af6790 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:42.856022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df9422-32eb-4cde-85a4-638cb2af6790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:42.886707 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f4df9422-32eb-4cde-85a4-638cb2af6790 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:42.889240 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f4df9422-32eb-4cde-85a4-638cb2af6790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:42.893450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:42.974850 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:43.008276 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67519d4e-ffd8-48a2-9913-8530c783bfc4 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:43.063855 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 82/167] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:43.064541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df9422-32eb-4cde-85a4-638cb2af6790 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:43.067405 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c95f9c-b103-4113-8071-ed6759389014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:43.068494 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c95f9c-b103-4113-8071-ed6759389014 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:43.069005 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c95f9c-b103-4113-8071-ed6759389014 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:43.069789 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c95f9c-b103-4113-8071-ed6759389014 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:43.070861 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c95f9c-b103-4113-8071-ed6759389014 None admin] RBAC: Adding query filter params (role.id=9cda4e2827dc4c2e87477bef8f319f92, scope.project.id=7cb636795baf4f7c85f61551f54b1f4e, user.id=8d5b327a728e436b857d5d3b3675bc2e) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:43.071408 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c95f9c-b103-4113-8071-ed6759389014 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:43.073826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c95f9c-b103-4113-8071-ed6759389014 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:43.081732 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 86/168] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:19:42 2022] GET /identity/v3/role_assignments?user.id=8d5b327a728e436b857d5d3b3675bc2e&scope.project.id=7cb636795baf4f7c85f61551f54b1f4e&role.id=9cda4e2827dc4c2e87477bef8f319f92 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:43.116483 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c95f9c-b103-4113-8071-ed6759389014 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:43.346138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df9422-32eb-4cde-85a4-638cb2af6790 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:43.378354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c95f9c-b103-4113-8071-ed6759389014 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:43.434675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091b8a2-a407-4a24-be9e-33226c6b9043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:43.435341 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091b8a2-a407-4a24-be9e-33226c6b9043 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:43.435622 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091b8a2-a407-4a24-be9e-33226c6b9043 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:43.435893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091b8a2-a407-4a24-be9e-33226c6b9043 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:43.454640 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcad24-ed90-4c09-8122-c8b48505b9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:43.455310 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcad24-ed90-4c09-8122-c8b48505b9ad None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:43.455593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcad24-ed90-4c09-8122-c8b48505b9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:43.455867 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcad24-ed90-4c09-8122-c8b48505b9ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:43.460671 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:43.472065 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091b8a2-a407-4a24-be9e-33226c6b9043 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:43.473031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:43.477333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-5091b8a2-a407-4a24-be9e-33226c6b9043 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:19:43.477333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:43.477333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:43.477333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:43.477333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:43.477333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:43.477333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:43.477333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:43.477333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:43.477333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:43.477333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:43.477333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:43.477333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:43.477333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:43.477333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:43.477333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:43.482498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:43.482498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:43.482498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:43.482498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:43.482498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:43.482498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:43.482498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:43.482498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:43.482498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:43.482498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:43.482498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:43.482498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:43.482498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:43.482498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:43.482498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:43.482498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:43.483355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:43.483355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:43.483355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:43.483355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:43.483355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:43.483355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:43.483355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:43.483355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:43.483355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:43.483355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:43.483355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:19:43.483355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:19:43.483355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 83/169] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:43.485209 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091b8a2-a407-4a24-be9e-33226c6b9043 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:43.486947 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcad24-ed90-4c09-8122-c8b48505b9ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:43.492746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-49dcad24-ed90-4c09-8122-c8b48505b9ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:19:43.492746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:43.492746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:43.492746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:43.492746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:43.492746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:43.492746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:43.492746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:43.492746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:43.492746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:43.492746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:43.492746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:43.492746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:43.492746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:43.492746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:43.492746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:43.493642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:43.493642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:43.493642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:43.493642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:43.493642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:43.493642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:43.493642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:43.493642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:43.493642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:43.493642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:43.493642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:43.493642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:43.493642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:43.493642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:43.493642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:43.493642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:43.502645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:43.502645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:43.502645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:43.502645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:43.502645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:43.502645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:43.502645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:43.502645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:43.502645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:43.502645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:43.502645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:19:43.502645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:19:43.505480 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 87/170] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:19:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:19:43.511336 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcad24-ed90-4c09-8122-c8b48505b9ad None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:43.661779 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcad24-ed90-4c09-8122-c8b48505b9ad None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:43.697068 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091b8a2-a407-4a24-be9e-33226c6b9043 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:43.707218 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a176d5-d2a2-4fe3-8a80-d5898719286a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:43.708273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a176d5-d2a2-4fe3-8a80-d5898719286a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:43.708749 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a176d5-d2a2-4fe3-8a80-d5898719286a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:43.709229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a176d5-d2a2-4fe3-8a80-d5898719286a None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:43.710206 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a176d5-d2a2-4fe3-8a80-d5898719286a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:43.710799 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a176d5-d2a2-4fe3-8a80-d5898719286a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:43.712457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a176d5-d2a2-4fe3-8a80-d5898719286a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:43.725224 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 88/171] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:19:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:43.732831 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a176d5-d2a2-4fe3-8a80-d5898719286a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:43.759702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af397e-31f4-4552-9977-2b695657fb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:43.760346 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af397e-31f4-4552-9977-2b695657fb6c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:43.760676 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af397e-31f4-4552-9977-2b695657fb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:43.761036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af397e-31f4-4552-9977-2b695657fb6c None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:43.761678 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af397e-31f4-4552-9977-2b695657fb6c None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:43.762012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af397e-31f4-4552-9977-2b695657fb6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:43.763433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af397e-31f4-4552-9977-2b695657fb6c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:43.778093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 84/172] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:19:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:43.783432 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af397e-31f4-4552-9977-2b695657fb6c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:43.810929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:43.858485 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a176d5-d2a2-4fe3-8a80-d5898719286a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:43.902313 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:43.933610 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbfbb0-7d78-425a-8f4a-cbaa29ad2195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:43.934672 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbfbb0-7d78-425a-8f4a-cbaa29ad2195 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:43.935147 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbfbb0-7d78-425a-8f4a-cbaa29ad2195 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:43.935708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbfbb0-7d78-425a-8f4a-cbaa29ad2195 None admin] PATH_INFO: `/v3/users/8d5b327a728e436b857d5d3b3675bc2e` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:43.938918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bbfbb0-7d78-425a-8f4a-cbaa29ad2195 None admin] RBAC: Authorizing `identity:get_user(user_id=8d5b327a728e436b857d5d3b3675bc2e)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:43.941059 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bbfbb0-7d78-425a-8f4a-cbaa29ad2195 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:43.943099 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 89/173] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:43 2022] GET /identity/v3/users/8d5b327a728e436b857d5d3b3675bc2e => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:43.945477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158313f-1c4a-411d-ab64-3326fa4a04bf None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:43.946072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158313f-1c4a-411d-ab64-3326fa4a04bf None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:43.946537 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158313f-1c4a-411d-ab64-3326fa4a04bf None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:43.949031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 90/174] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:43.950365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57d41a-c5a0-49da-ab31-1ca7caaa1053 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:43.950952 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57d41a-c5a0-49da-ab31-1ca7caaa1053 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:43.951899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57d41a-c5a0-49da-ab31-1ca7caaa1053 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:43.953675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 91/175] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:43.954590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57d41a-c5a0-49da-ab31-1ca7caaa1053 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:43.969546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af397e-31f4-4552-9977-2b695657fb6c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:44.005873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10cdfe-ff3b-4ed3-bbf2-f466001d023f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:44.006660 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10cdfe-ff3b-4ed3-bbf2-f466001d023f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:44.006940 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10cdfe-ff3b-4ed3-bbf2-f466001d023f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:44.007216 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10cdfe-ff3b-4ed3-bbf2-f466001d023f None admin] PATH_INFO: `/v3/users/8d5b327a728e436b857d5d3b3675bc2e` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:44.008514 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10cdfe-ff3b-4ed3-bbf2-f466001d023f None admin] RBAC: Authorizing `identity:get_user(user_id=8d5b327a728e436b857d5d3b3675bc2e)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:44.010023 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10cdfe-ff3b-4ed3-bbf2-f466001d023f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:44.014041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 85/176] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:43 2022] GET /identity/v3/users/8d5b327a728e436b857d5d3b3675bc2e => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:44.014041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2a79b-ee47-43aa-9813-5db3504b7934 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:44.014041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2a79b-ee47-43aa-9813-5db3504b7934 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:44.014390 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2a79b-ee47-43aa-9813-5db3504b7934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:44.033531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-5cb2a79b-ee47-43aa-9813-5db3504b7934 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:44.034881 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5cb2a79b-ee47-43aa-9813-5db3504b7934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:44.077401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57d41a-c5a0-49da-ab31-1ca7caaa1053 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:44.109602 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:44.117673 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 86/177] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:44.124868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2893d0f-e4cd-4e41-8821-53d98bcfb7a1 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:44.124868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2893d0f-e4cd-4e41-8821-53d98bcfb7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:44.124868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2893d0f-e4cd-4e41-8821-53d98bcfb7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:44.126229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66984d9d-c6fc-4394-b285-97ff3801c53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:44.126904 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66984d9d-c6fc-4394-b285-97ff3801c53b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:44.127189 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66984d9d-c6fc-4394-b285-97ff3801c53b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:44.127446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66984d9d-c6fc-4394-b285-97ff3801c53b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:44.145971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a2893d0f-e4cd-4e41-8821-53d98bcfb7a1 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:44.147425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a2893d0f-e4cd-4e41-8821-53d98bcfb7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:44.211490 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66984d9d-c6fc-4394-b285-97ff3801c53b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:44.212896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66984d9d-c6fc-4394-b285-97ff3801c53b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:44.214360 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:44.228104 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 87/178] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:44.229229 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175115d1-c4e6-49e3-b310-08b8fd0c7c9f None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:44.229514 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175115d1-c4e6-49e3-b310-08b8fd0c7c9f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:44.230449 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175115d1-c4e6-49e3-b310-08b8fd0c7c9f None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:44.231290 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 88/179] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:44.235938 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175115d1-c4e6-49e3-b310-08b8fd0c7c9f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:44.322759 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 92/180] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:19:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:19:44.324063 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66984d9d-c6fc-4394-b285-97ff3801c53b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:44.342027 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175115d1-c4e6-49e3-b310-08b8fd0c7c9f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:44.355697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:44.377999 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd44813-7397-4b54-b73f-7f08cc2d8cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:44.378622 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd44813-7397-4b54-b73f-7f08cc2d8cce None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:44.378907 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd44813-7397-4b54-b73f-7f08cc2d8cce None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:44.379183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd44813-7397-4b54-b73f-7f08cc2d8cce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:44.426529 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66984d9d-c6fc-4394-b285-97ff3801c53b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:44.438629 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:44.450117 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd44813-7397-4b54-b73f-7f08cc2d8cce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:44.451666 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd44813-7397-4b54-b73f-7f08cc2d8cce None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:44.460871 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb358d6c-35d6-4edf-a5b7-9477392830ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:44.461557 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb358d6c-35d6-4edf-a5b7-9477392830ae None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:44.461791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb358d6c-35d6-4edf-a5b7-9477392830ae None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:44.462095 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb358d6c-35d6-4edf-a5b7-9477392830ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:44.463253 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb358d6c-35d6-4edf-a5b7-9477392830ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:44.464729 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb358d6c-35d6-4edf-a5b7-9477392830ae None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:44.466390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 93/181] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:19:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:44.471142 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01724a-3c60-425c-9683-6842759323f5 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:44.471142 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01724a-3c60-425c-9683-6842759323f5 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:44.471142 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01724a-3c60-425c-9683-6842759323f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:44.485838 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-af01724a-3c60-425c-9683-6842759323f5 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:44.487082 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-af01724a-3c60-425c-9683-6842759323f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:44.554094 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 94/182] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:44.555417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d0ed8-0eca-464a-a5f8-164dc836f796 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:44.555735 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d0ed8-0eca-464a-a5f8-164dc836f796 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:44.555976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d0ed8-0eca-464a-a5f8-164dc836f796 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:44.561863 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 95/183] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:44.561863 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d0ed8-0eca-464a-a5f8-164dc836f796 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:44.562665 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 89/184] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:19:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:19:44.563579 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd44813-7397-4b54-b73f-7f08cc2d8cce None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:44.599583 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:44.654759 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:44.664748 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd44813-7397-4b54-b73f-7f08cc2d8cce None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:44.677578 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d0ed8-0eca-464a-a5f8-164dc836f796 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:44.700029 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8ccc9-5e7a-4a5a-8efc-f5d3fe36f961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:44.700681 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8ccc9-5e7a-4a5a-8efc-f5d3fe36f961 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:44.700958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8ccc9-5e7a-4a5a-8efc-f5d3fe36f961 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:44.701230 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8ccc9-5e7a-4a5a-8efc-f5d3fe36f961 None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:44.702644 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d8ccc9-5e7a-4a5a-8efc-f5d3fe36f961 None admin] RBAC: Authorizing `identity:get_project(project_id=44e0498d623545e68ea89aa6de0a8be7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:44.704234 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d8ccc9-5e7a-4a5a-8efc-f5d3fe36f961 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:44.706363 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 90/185] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:44 2022] GET /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:44.707158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8ccc9-5e7a-4a5a-8efc-f5d3fe36f961 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:44.712929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165b027-2dc6-4cba-9f3b-79386fc7c7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:44.713548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165b027-2dc6-4cba-9f3b-79386fc7c7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:44.713824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165b027-2dc6-4cba-9f3b-79386fc7c7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:44.714131 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165b027-2dc6-4cba-9f3b-79386fc7c7c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:44.724480 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165b027-2dc6-4cba-9f3b-79386fc7c7c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:44.730040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-c165b027-2dc6-4cba-9f3b-79386fc7c7c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:19:44.730040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:44.730040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:44.730040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:44.730040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:44.730040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:44.730040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:44.730040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:44.730040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:44.730040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:44.730040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:44.730040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:44.730040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:44.730040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:44.730040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:44.730040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:44.731061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:44.731061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:44.731061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:44.731061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:44.731061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:44.731061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:44.731061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:44.731061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:44.731061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:44.731061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:44.731061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:44.731061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:44.731061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:44.731061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:44.731061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:44.731061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:44.731864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:44.731864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:44.731864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:44.731864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:44.731864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:44.731864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:44.731864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:44.731864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:44.731864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:44.731864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:44.731864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:19:44.731864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:19:44.731864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 96/186] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:19:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:44.732578 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1485d-73d7-414e-8513-034daab4304f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:44.732763 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1485d-73d7-414e-8513-034daab4304f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:44.733038 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1485d-73d7-414e-8513-034daab4304f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:44.751059 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c8a1485d-73d7-414e-8513-034daab4304f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:44.752255 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c8a1485d-73d7-414e-8513-034daab4304f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:44.816618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8ccc9-5e7a-4a5a-8efc-f5d3fe36f961 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:44.822170 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 97/187] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:44.823141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1485d-73d7-414e-8513-034daab4304f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:44.842460 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:44.860344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750c02d-1988-4248-9028-41ab70584bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:44.860973 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750c02d-1988-4248-9028-41ab70584bbb None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:44.861255 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750c02d-1988-4248-9028-41ab70584bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:44.861529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750c02d-1988-4248-9028-41ab70584bbb None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:44.862252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750c02d-1988-4248-9028-41ab70584bbb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:44.863704 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750c02d-1988-4248-9028-41ab70584bbb None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:44.885434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 91/188] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Aug 1 15:19:44 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:44.886458 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750c02d-1988-4248-9028-41ab70584bbb None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:44.904504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:44.939987 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1485d-73d7-414e-8513-034daab4304f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:44.982143 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcde3f-6280-4e74-b2e7-6f07331c90df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:44.982726 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcde3f-6280-4e74-b2e7-6f07331c90df None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:44.983008 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcde3f-6280-4e74-b2e7-6f07331c90df None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:44.983280 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcde3f-6280-4e74-b2e7-6f07331c90df None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:44.984528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dcde3f-6280-4e74-b2e7-6f07331c90df None admin] RBAC: Authorizing `identity:get_project(project_id=44e0498d623545e68ea89aa6de0a8be7)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:44.986126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dcde3f-6280-4e74-b2e7-6f07331c90df None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:44.988060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 98/189] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:44 2022] GET /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:44.988753 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcde3f-6280-4e74-b2e7-6f07331c90df None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:45.014459 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750c02d-1988-4248-9028-41ab70584bbb None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:45.063297 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915fa8d-3605-4d9e-bdce-4e4b7a9e0b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:45.063968 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915fa8d-3605-4d9e-bdce-4e4b7a9e0b8e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:45.064337 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915fa8d-3605-4d9e-bdce-4e4b7a9e0b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:45.064618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915fa8d-3605-4d9e-bdce-4e4b7a9e0b8e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:45.076879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e915fa8d-3605-4d9e-bdce-4e4b7a9e0b8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:45.082419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-e915fa8d-3605-4d9e-bdce-4e4b7a9e0b8e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:19:45.082419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:45.082419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:45.082419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:45.082419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:45.082419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:45.082419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:45.082419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:45.082419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:45.082419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:45.082419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:45.082419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:45.082419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:45.082419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:45.082419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:45.082419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:45.083414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:45.083414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:45.083414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:45.083414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:45.083414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:45.083414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:45.083414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:45.083414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:45.083414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:45.083414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:45.083414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:45.083414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:45.083414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:45.083414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:45.083414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:45.083414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:45.084215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:45.084215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:45.084215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:45.084215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:45.084215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:45.084215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:45.084215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:45.084215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:45.084215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:45.084215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:45.084215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:19:45.084215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:19:45.084215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 92/190] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:19:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:45.084871 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915fa8d-3605-4d9e-bdce-4e4b7a9e0b8e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:45.120007 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:45.143899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcde3f-6280-4e74-b2e7-6f07331c90df None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:45.186770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:45.197673 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7e644-ddf3-4275-b7c0-4fa6bd45b428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:45.198464 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7e644-ddf3-4275-b7c0-4fa6bd45b428 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:45.199190 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7e644-ddf3-4275-b7c0-4fa6bd45b428 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:45.199474 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7e644-ddf3-4275-b7c0-4fa6bd45b428 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:45.200576 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e7e644-ddf3-4275-b7c0-4fa6bd45b428 None admin] RBAC: Adding query filter params (role.id=73e29e4f51e6469facf94aaa698c5cfe, scope.project.id=44e0498d623545e68ea89aa6de0a8be7, user.id=8d5b327a728e436b857d5d3b3675bc2e) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:45.200892 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e7e644-ddf3-4275-b7c0-4fa6bd45b428 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:45.202962 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e7e644-ddf3-4275-b7c0-4fa6bd45b428 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:45.211106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 99/191] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:19:44 2022] GET /identity/v3/role_assignments?user.id=8d5b327a728e436b857d5d3b3675bc2e&scope.project.id=44e0498d623545e68ea89aa6de0a8be7&role.id=73e29e4f51e6469facf94aaa698c5cfe => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:45.212302 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7e644-ddf3-4275-b7c0-4fa6bd45b428 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:45.221617 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915fa8d-3605-4d9e-bdce-4e4b7a9e0b8e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:45.277015 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0b569-85bc-4bc7-a99f-df1d9a1f9137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:45.277685 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0b569-85bc-4bc7-a99f-df1d9a1f9137 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:45.278039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0b569-85bc-4bc7-a99f-df1d9a1f9137 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:45.278324 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0b569-85bc-4bc7-a99f-df1d9a1f9137 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:45.279343 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0b569-85bc-4bc7-a99f-df1d9a1f9137 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:45.279656 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0b569-85bc-4bc7-a99f-df1d9a1f9137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:45.281965 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0b569-85bc-4bc7-a99f-df1d9a1f9137 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:45.294939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 93/192] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:45.296081 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0b569-85bc-4bc7-a99f-df1d9a1f9137 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:45.365605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7e644-ddf3-4275-b7c0-4fa6bd45b428 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:45.418005 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2205c-54f8-4225-bab5-ec95507c328a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:45.422622 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2205c-54f8-4225-bab5-ec95507c328a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:45.422972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2205c-54f8-4225-bab5-ec95507c328a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:45.423247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2205c-54f8-4225-bab5-ec95507c328a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:45.437114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0b569-85bc-4bc7-a99f-df1d9a1f9137 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:45.437986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2205c-54f8-4225-bab5-ec95507c328a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:45.445475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-04c2205c-54f8-4225-bab5-ec95507c328a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:19:45.445475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:45.445475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:45.445475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:45.445475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:45.445475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:45.445475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:45.445475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:45.445475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:45.445475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:45.445475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:45.445475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:45.445475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:45.445475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:45.445475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:45.445475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:45.446544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:45.446544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:45.446544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:45.446544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:45.446544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:45.446544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:45.446544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:45.446544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:45.446544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:45.446544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:45.446544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:45.446544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:45.446544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:45.446544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:45.446544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:45.446544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:45.447335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:45.447335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:45.447335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:45.447335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:45.447335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:45.447335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:45.447335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:45.447335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:45.447335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:45.447335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:45.447335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:19:45.447335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:19:45.447335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 100/193] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:45.448008 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2205c-54f8-4225-bab5-ec95507c328a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:45.464892 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:45.507957 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920732c-f97f-4b62-adbc-dc10674a5a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:45.508834 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920732c-f97f-4b62-adbc-dc10674a5a8b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:45.509251 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920732c-f97f-4b62-adbc-dc10674a5a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:45.509625 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920732c-f97f-4b62-adbc-dc10674a5a8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:45.511405 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920732c-f97f-4b62-adbc-dc10674a5a8b None admin] RBAC: Adding query filter params (role.id=9cda4e2827dc4c2e87477bef8f319f92, scope.project.id=44e0498d623545e68ea89aa6de0a8be7, user.id=8d5b327a728e436b857d5d3b3675bc2e) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:45.511996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920732c-f97f-4b62-adbc-dc10674a5a8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:45.514944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920732c-f97f-4b62-adbc-dc10674a5a8b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:45.520402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:45.532817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 94/194] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:19:45 2022] GET /identity/v3/role_assignments?user.id=8d5b327a728e436b857d5d3b3675bc2e&scope.project.id=44e0498d623545e68ea89aa6de0a8be7&role.id=9cda4e2827dc4c2e87477bef8f319f92 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:45.534433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920732c-f97f-4b62-adbc-dc10674a5a8b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:45.604984 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2205c-54f8-4225-bab5-ec95507c328a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:45.647838 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8b882-2348-4471-a1bd-e855c0fd4556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:45.648907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8b882-2348-4471-a1bd-e855c0fd4556 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:45.649389 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8b882-2348-4471-a1bd-e855c0fd4556 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:45.649857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8b882-2348-4471-a1bd-e855c0fd4556 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:45.650870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8b882-2348-4471-a1bd-e855c0fd4556 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:45.651938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8b882-2348-4471-a1bd-e855c0fd4556 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:45.653880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8b882-2348-4471-a1bd-e855c0fd4556 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:45.664188 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 101/195] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:19:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:45.665583 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8b882-2348-4471-a1bd-e855c0fd4556 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:45.756111 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:45.795101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8b882-2348-4471-a1bd-e855c0fd4556 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:45.837852 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e14c130-1ad5-4816-9b2e-52cb12914f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:45.838901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14c130-1ad5-4816-9b2e-52cb12914f00 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:45.839767 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14c130-1ad5-4816-9b2e-52cb12914f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:45.840265 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14c130-1ad5-4816-9b2e-52cb12914f00 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:45.841188 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e14c130-1ad5-4816-9b2e-52cb12914f00 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:45.841829 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e14c130-1ad5-4816-9b2e-52cb12914f00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:45.843508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e14c130-1ad5-4816-9b2e-52cb12914f00 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:45.854131 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 102/196] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:19:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:45.855560 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e14c130-1ad5-4816-9b2e-52cb12914f00 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:45.859455 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920732c-f97f-4b62-adbc-dc10674a5a8b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:45.970627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30940f-5e0c-4088-b55d-cfdb5750a6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:45.971795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30940f-5e0c-4088-b55d-cfdb5750a6af None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:45.972354 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30940f-5e0c-4088-b55d-cfdb5750a6af None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:45.972851 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30940f-5e0c-4088-b55d-cfdb5750a6af None admin] PATH_INFO: `/v3/users/168e395a154c44fd826effd7718122ac` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:45.981342 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e14c130-1ad5-4816-9b2e-52cb12914f00 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:46.002724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df30940f-5e0c-4088-b55d-cfdb5750a6af None admin] RBAC: Authorizing `identity:get_user(user_id=168e395a154c44fd826effd7718122ac)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:46.005694 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df30940f-5e0c-4088-b55d-cfdb5750a6af None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:46.016746 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 95/197] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:45 2022] GET /identity/v3/users/168e395a154c44fd826effd7718122ac => generated 283 bytes in 477 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:46.018462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30940f-5e0c-4088-b55d-cfdb5750a6af None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:46.021920 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc6916-94df-4d45-afa5-d2f7dc22afc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:46.023010 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc6916-94df-4d45-afa5-d2f7dc22afc2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:46.023492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc6916-94df-4d45-afa5-d2f7dc22afc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:46.024036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc6916-94df-4d45-afa5-d2f7dc22afc2 None admin] PATH_INFO: `/v3/users/168e395a154c44fd826effd7718122ac` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:46.026526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc6916-94df-4d45-afa5-d2f7dc22afc2 None admin] RBAC: Authorizing `identity:get_user(user_id=168e395a154c44fd826effd7718122ac)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:46.028266 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc6916-94df-4d45-afa5-d2f7dc22afc2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:46.030149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 103/198] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:45 2022] GET /identity/v3/users/168e395a154c44fd826effd7718122ac => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:46.035540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc6916-94df-4d45-afa5-d2f7dc22afc2 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:46.041131 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:46.110989 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:46.162172 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30940f-5e0c-4088-b55d-cfdb5750a6af None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:46.174312 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc6916-94df-4d45-afa5-d2f7dc22afc2 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:46.207860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953d55c-fa3b-4361-b5b6-827019708b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:46.208447 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953d55c-fa3b-4361-b5b6-827019708b43 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:46.208730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953d55c-fa3b-4361-b5b6-827019708b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:46.209006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953d55c-fa3b-4361-b5b6-827019708b43 None admin] PATH_INFO: `/v3/users/77510eb658e346728640f3cbede2912a` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:46.210260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953d55c-fa3b-4361-b5b6-827019708b43 None admin] RBAC: Authorizing `identity:get_user(user_id=77510eb658e346728640f3cbede2912a)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:46.211727 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953d55c-fa3b-4361-b5b6-827019708b43 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:46.213794 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 96/199] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:46 2022] GET /identity/v3/users/77510eb658e346728640f3cbede2912a => generated 289 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:46.214616 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953d55c-fa3b-4361-b5b6-827019708b43 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:46.219451 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18cda6-03c4-4772-a87c-b6e6a149f093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:46.220399 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18cda6-03c4-4772-a87c-b6e6a149f093 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:46.220985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18cda6-03c4-4772-a87c-b6e6a149f093 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:46.221470 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18cda6-03c4-4772-a87c-b6e6a149f093 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:46.310323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18cda6-03c4-4772-a87c-b6e6a149f093 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:46.312531 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18cda6-03c4-4772-a87c-b6e6a149f093 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:46.315361 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:46.339694 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953d55c-fa3b-4361-b5b6-827019708b43 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:46.368558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:46.380365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c566d1-9c14-4bc1-8030-ec2d7a3b7677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:46.380868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c566d1-9c14-4bc1-8030-ec2d7a3b7677 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:46.381196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c566d1-9c14-4bc1-8030-ec2d7a3b7677 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:46.381484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c566d1-9c14-4bc1-8030-ec2d7a3b7677 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:46.433457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 104/200] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:19:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 15:19:46.434674 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18cda6-03c4-4772-a87c-b6e6a149f093 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:46.459333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c566d1-9c14-4bc1-8030-ec2d7a3b7677 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:46.460975 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c566d1-9c14-4bc1-8030-ec2d7a3b7677 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:46.543899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18cda6-03c4-4772-a87c-b6e6a149f093 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:46.545836 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:46.582699 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 97/201] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:19:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:19:46.582699 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c566d1-9c14-4bc1-8030-ec2d7a3b7677 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:46.586144 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1396dc-d5ca-489d-ac1a-7a284933d091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:46.587005 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1396dc-d5ca-489d-ac1a-7a284933d091 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:46.587486 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1396dc-d5ca-489d-ac1a-7a284933d091 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:46.587952 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1396dc-d5ca-489d-ac1a-7a284933d091 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:46.610280 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:46.663481 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1396dc-d5ca-489d-ac1a-7a284933d091 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:46.665578 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1396dc-d5ca-489d-ac1a-7a284933d091 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:46.692536 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c566d1-9c14-4bc1-8030-ec2d7a3b7677 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:46.731672 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e84b6-a6e0-4cd1-a24b-ece9af90a8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:46.732383 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e84b6-a6e0-4cd1-a24b-ece9af90a8da None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:46.732669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e84b6-a6e0-4cd1-a24b-ece9af90a8da None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:46.732945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e84b6-a6e0-4cd1-a24b-ece9af90a8da None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:46.734270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e84b6-a6e0-4cd1-a24b-ece9af90a8da None admin] RBAC: Authorizing `identity:get_project(project_id=d659ef9d8a7e447cb15feaa6be1b2787)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:46.736049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e84b6-a6e0-4cd1-a24b-ece9af90a8da None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:46.738696 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 98/202] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:46 2022] GET /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:46.739371 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e84b6-a6e0-4cd1-a24b-ece9af90a8da None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:46.775523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 105/203] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:19:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:19:46.777792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1396dc-d5ca-489d-ac1a-7a284933d091 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:46.780449 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:46.848108 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:46.855888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e84b6-a6e0-4cd1-a24b-ece9af90a8da None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:46.892164 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1396dc-d5ca-489d-ac1a-7a284933d091 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:46.894226 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e13b8-e2c0-40a4-93ae-6bb52c263ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:46.896056 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e13b8-e2c0-40a4-93ae-6bb52c263ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:46.896527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e13b8-e2c0-40a4-93ae-6bb52c263ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:46.897025 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e13b8-e2c0-40a4-93ae-6bb52c263ee3 None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:46.898501 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e13b8-e2c0-40a4-93ae-6bb52c263ee3 None admin] RBAC: Authorizing `identity:get_project(project_id=d659ef9d8a7e447cb15feaa6be1b2787)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:46.900175 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e13b8-e2c0-40a4-93ae-6bb52c263ee3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:46.903682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 99/204] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:46 2022] GET /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:46.905200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e13b8-e2c0-40a4-93ae-6bb52c263ee3 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:46.933870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3f8f6-2ce1-42dd-97fe-da72cbe269d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:46.934807 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3f8f6-2ce1-42dd-97fe-da72cbe269d6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:46.935218 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3f8f6-2ce1-42dd-97fe-da72cbe269d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:46.935580 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3f8f6-2ce1-42dd-97fe-da72cbe269d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:46.936845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3f8f6-2ce1-42dd-97fe-da72cbe269d6 None admin] RBAC: Adding query filter params (role.id=73e29e4f51e6469facf94aaa698c5cfe, scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787, user.id=168e395a154c44fd826effd7718122ac) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:46.937088 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3f8f6-2ce1-42dd-97fe-da72cbe269d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:46.938798 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3f8f6-2ce1-42dd-97fe-da72cbe269d6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:46.946043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 106/205] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:19:46 2022] GET /identity/v3/role_assignments?user.id=168e395a154c44fd826effd7718122ac&scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787&role.id=73e29e4f51e6469facf94aaa698c5cfe => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:46.951174 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3f8f6-2ce1-42dd-97fe-da72cbe269d6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:47.037662 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e13b8-e2c0-40a4-93ae-6bb52c263ee3 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:47.081461 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9166ee-4514-4d21-ad6b-11d6b5bf5d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:47.082167 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9166ee-4514-4d21-ad6b-11d6b5bf5d84 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:47.082454 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9166ee-4514-4d21-ad6b-11d6b5bf5d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:47.082731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9166ee-4514-4d21-ad6b-11d6b5bf5d84 None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:47.084825 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9166ee-4514-4d21-ad6b-11d6b5bf5d84 None admin] RBAC: Authorizing `identity:get_project(project_id=44e0498d623545e68ea89aa6de0a8be7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:47.086744 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9166ee-4514-4d21-ad6b-11d6b5bf5d84 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:47.089768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 100/206] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:46 2022] GET /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:47.095594 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9166ee-4514-4d21-ad6b-11d6b5bf5d84 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:47.110361 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:47.156108 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3f8f6-2ce1-42dd-97fe-da72cbe269d6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:47.197011 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:47.207514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87ede1-46c3-4b0c-8004-cc686dcd9ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:47.208412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87ede1-46c3-4b0c-8004-cc686dcd9ceb None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:47.208884 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87ede1-46c3-4b0c-8004-cc686dcd9ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:47.209302 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87ede1-46c3-4b0c-8004-cc686dcd9ceb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:47.211040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87ede1-46c3-4b0c-8004-cc686dcd9ceb None admin] RBAC: Adding query filter params (role.id=7bdfa43dc1d046f7965ffe52b74bc58f, scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787, user.id=168e395a154c44fd826effd7718122ac) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:47.211518 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87ede1-46c3-4b0c-8004-cc686dcd9ceb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:47.213131 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87ede1-46c3-4b0c-8004-cc686dcd9ceb None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:47.220645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 107/207] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:19:46 2022] GET /identity/v3/role_assignments?user.id=168e395a154c44fd826effd7718122ac&scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787&role.id=7bdfa43dc1d046f7965ffe52b74bc58f => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:47.248389 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9e249-def8-4d6a-be12-9ec22686b8f6 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:47.248732 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9e249-def8-4d6a-be12-9ec22686b8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:47.248952 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9e249-def8-4d6a-be12-9ec22686b8f6 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:47.250617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 108/208] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:47.256739 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b418a-5527-4f59-b851-3adaa8a4aa65 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:47.257027 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b418a-5527-4f59-b851-3adaa8a4aa65 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:47.257312 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b418a-5527-4f59-b851-3adaa8a4aa65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:47.277170 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-ff5b418a-5527-4f59-b851-3adaa8a4aa65 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:47.278447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ff5b418a-5527-4f59-b851-3adaa8a4aa65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:47.292776 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9166ee-4514-4d21-ad6b-11d6b5bf5d84 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:47.359329 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c12362-80eb-4d28-a62e-2e36f8e2689d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:47.359988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c12362-80eb-4d28-a62e-2e36f8e2689d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:47.362665 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c12362-80eb-4d28-a62e-2e36f8e2689d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:47.362890 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c12362-80eb-4d28-a62e-2e36f8e2689d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:47.363726 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c12362-80eb-4d28-a62e-2e36f8e2689d None admin] RBAC: Adding query filter params (role.id=fdbe7b923fed4101826cfb93a92e3fbf, scope.project.id=44e0498d623545e68ea89aa6de0a8be7, user.id=77510eb658e346728640f3cbede2912a) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:47.365282 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c12362-80eb-4d28-a62e-2e36f8e2689d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:47.368273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c12362-80eb-4d28-a62e-2e36f8e2689d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:47.381461 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 101/209] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:19:47 2022] GET /identity/v3/role_assignments?user.id=77510eb658e346728640f3cbede2912a&scope.project.id=44e0498d623545e68ea89aa6de0a8be7&role.id=fdbe7b923fed4101826cfb93a92e3fbf => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:47.411837 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 109/210] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:19:47.448409 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c12362-80eb-4d28-a62e-2e36f8e2689d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:47.497968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ed05f-fd02-4631-ad53-77283527a6c5 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:47.498294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ed05f-fd02-4631-ad53-77283527a6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:47.498535 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ed05f-fd02-4631-ad53-77283527a6c5 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:47.500724 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 110/211] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:19:47.508738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867163c-d7a6-4b08-bfda-54cb8b1783ae None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:47.509179 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867163c-d7a6-4b08-bfda-54cb8b1783ae None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:47.509576 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867163c-d7a6-4b08-bfda-54cb8b1783ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:47.540025 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-3867163c-d7a6-4b08-bfda-54cb8b1783ae None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:47.541545 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3867163c-d7a6-4b08-bfda-54cb8b1783ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:47.615181 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:47.620169 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:47.628180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c12362-80eb-4d28-a62e-2e36f8e2689d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:47.630935 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 111/212] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:19:47.658534 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867163c-d7a6-4b08-bfda-54cb8b1783ae None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:47.666453 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309defa-59e8-49da-9ac5-63d4685f1486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:47.667669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309defa-59e8-49da-9ac5-63d4685f1486 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:47.667954 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309defa-59e8-49da-9ac5-63d4685f1486 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:47.668235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309defa-59e8-49da-9ac5-63d4685f1486 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:47.685022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d309defa-59e8-49da-9ac5-63d4685f1486 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:47.692181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-d309defa-59e8-49da-9ac5-63d4685f1486 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:19:47.692181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:47.692181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:47.692181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:47.692181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:47.692181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:47.692181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:47.692181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:47.692181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:47.692181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:47.692181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:47.692181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:47.692181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:47.692181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:47.692181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:47.692181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:47.693148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:47.693148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:47.693148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:47.693148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:47.693148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:47.693148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:47.693148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:47.693148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:47.693148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:47.693148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:47.693148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:47.693148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:47.693148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:47.693148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:47.693148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:47.693148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:47.694715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:47.694715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:47.694715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:47.694715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:47.694715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:47.694715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:47.694715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:47.694715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:47.694715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:47.694715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:47.694715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:19:47.694715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:19:47.694715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 102/213] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:47.700576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309defa-59e8-49da-9ac5-63d4685f1486 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:47.779628 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867163c-d7a6-4b08-bfda-54cb8b1783ae None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:47.817783 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309defa-59e8-49da-9ac5-63d4685f1486 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:47.818582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2adc05-859d-40b5-a9f6-2c958c84d4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:47.819845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2adc05-859d-40b5-a9f6-2c958c84d4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:47.820317 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2adc05-859d-40b5-a9f6-2c958c84d4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:47.820780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2adc05-859d-40b5-a9f6-2c958c84d4b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:47.822266 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2adc05-859d-40b5-a9f6-2c958c84d4b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:47.824564 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2adc05-859d-40b5-a9f6-2c958c84d4b2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:47.826829 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 112/214] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:19:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:47.834788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2adc05-859d-40b5-a9f6-2c958c84d4b2 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:47.862088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c854d-4448-4d92-bb83-280a29c08f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:47.862712 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c854d-4448-4d92-bb83-280a29c08f6b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:47.862999 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c854d-4448-4d92-bb83-280a29c08f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:47.863280 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c854d-4448-4d92-bb83-280a29c08f6b None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:47.864018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c854d-4448-4d92-bb83-280a29c08f6b None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:47.864331 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c854d-4448-4d92-bb83-280a29c08f6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:47.866052 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c854d-4448-4d92-bb83-280a29c08f6b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:47.876418 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 103/215] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:19:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:47.880903 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c854d-4448-4d92-bb83-280a29c08f6b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:47.903998 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:47.911316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:47.958004 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2adc05-859d-40b5-a9f6-2c958c84d4b2 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:47.998008 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6c836-7118-4d85-9c4d-d29cf96c28b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:47.999669 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6c836-7118-4d85-9c4d-d29cf96c28b4 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:48.000317 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6c836-7118-4d85-9c4d-d29cf96c28b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:48.000790 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6c836-7118-4d85-9c4d-d29cf96c28b4 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:48.001312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c854d-4448-4d92-bb83-280a29c08f6b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:48.001953 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6c836-7118-4d85-9c4d-d29cf96c28b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:48.004937 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6c836-7118-4d85-9c4d-d29cf96c28b4 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:48.034685 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 113/216] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Aug 1 15:19:47 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:19:48.046247 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60bc38-ff36-43ba-a0d7-89a4143cf0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:48.047303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60bc38-ff36-43ba-a0d7-89a4143cf0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:48.047863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60bc38-ff36-43ba-a0d7-89a4143cf0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:48.048339 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60bc38-ff36-43ba-a0d7-89a4143cf0c2 None admin] PATH_INFO: `/v3/users/9d56df6133fd4e0d83b6090aa2abdcc5` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:48.070423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60bc38-ff36-43ba-a0d7-89a4143cf0c2 None admin] RBAC: Authorizing `identity:get_user(user_id=9d56df6133fd4e0d83b6090aa2abdcc5)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:48.070423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60bc38-ff36-43ba-a0d7-89a4143cf0c2 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:48.070681 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 104/217] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:47 2022] GET /identity/v3/users/9d56df6133fd4e0d83b6090aa2abdcc5 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:48.077995 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6c836-7118-4d85-9c4d-d29cf96c28b4 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:48.194276 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6c836-7118-4d85-9c4d-d29cf96c28b4 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:48.215237 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:48.238306 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49514de0-8379-4917-9c7e-cf3b21853640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:48.238943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49514de0-8379-4917-9c7e-cf3b21853640 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:48.239227 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49514de0-8379-4917-9c7e-cf3b21853640 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:48.239502 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49514de0-8379-4917-9c7e-cf3b21853640 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:48.314211 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49514de0-8379-4917-9c7e-cf3b21853640 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:48.315760 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49514de0-8379-4917-9c7e-cf3b21853640 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:48.428003 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 114/218] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:19:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:19:48.434600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60bc38-ff36-43ba-a0d7-89a4143cf0c2 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:48.530864 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:48.560852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60bc38-ff36-43ba-a0d7-89a4143cf0c2 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:48.599491 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844782a-70d9-4f7e-a3e7-3e9640d0cf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:48.600153 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844782a-70d9-4f7e-a3e7-3e9640d0cf56 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:48.600435 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844782a-70d9-4f7e-a3e7-3e9640d0cf56 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:48.600707 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844782a-70d9-4f7e-a3e7-3e9640d0cf56 None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:48.601864 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4844782a-70d9-4f7e-a3e7-3e9640d0cf56 None admin] RBAC: Authorizing `identity:get_project(project_id=d659ef9d8a7e447cb15feaa6be1b2787)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:48.603412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4844782a-70d9-4f7e-a3e7-3e9640d0cf56 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:48.605945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 105/219] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:48 2022] GET /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:48.613855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49514de0-8379-4917-9c7e-cf3b21853640 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:48.639450 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:48.742769 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49514de0-8379-4917-9c7e-cf3b21853640 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:48.785371 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a855f8-00a3-46dc-a86a-45a89706ec3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:48.786129 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a855f8-00a3-46dc-a86a-45a89706ec3e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:48.786288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a855f8-00a3-46dc-a86a-45a89706ec3e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:48.786435 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a855f8-00a3-46dc-a86a-45a89706ec3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:48.787232 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a855f8-00a3-46dc-a86a-45a89706ec3e None admin] RBAC: Adding query filter params (role.id=9cda4e2827dc4c2e87477bef8f319f92, scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787, user.id=9d56df6133fd4e0d83b6090aa2abdcc5) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:48.787539 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a855f8-00a3-46dc-a86a-45a89706ec3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:48.788979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a855f8-00a3-46dc-a86a-45a89706ec3e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:48.799628 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 115/220] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:19:48 2022] GET /identity/v3/role_assignments?user.id=9d56df6133fd4e0d83b6090aa2abdcc5&scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787&role.id=9cda4e2827dc4c2e87477bef8f319f92 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:49.499215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f03ac-c460-46ca-8b27-19af6d3f91b4 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:49.499899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f03ac-c460-46ca-8b27-19af6d3f91b4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:49.499899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f03ac-c460-46ca-8b27-19af6d3f91b4 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:49.501500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 106/221] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:49.506454 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6df45-b112-4309-8eec-88332d3b2530 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:49.506743 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6df45-b112-4309-8eec-88332d3b2530 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:49.506998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6df45-b112-4309-8eec-88332d3b2530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:49.530859 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d5d6df45-b112-4309-8eec-88332d3b2530 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:49.532116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d5d6df45-b112-4309-8eec-88332d3b2530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:49.551888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e5698-287e-43df-bd18-6c5395d09a7b None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:49.552500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e5698-287e-43df-bd18-6c5395d09a7b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:49.552869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e5698-287e-43df-bd18-6c5395d09a7b None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:49.554458 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 107/222] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:49.560482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729db301-ca87-40c8-be3c-13ff66cf294c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:49.560856 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729db301-ca87-40c8-be3c-13ff66cf294c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:49.561231 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729db301-ca87-40c8-be3c-13ff66cf294c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:49.605521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-729db301-ca87-40c8-be3c-13ff66cf294c None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:49.606751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-729db301-ca87-40c8-be3c-13ff66cf294c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:49.633481 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 116/223] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:19:49.672109 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6df45-b112-4309-8eec-88332d3b2530 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:49.683681 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:49.770368 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 108/224] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:19:49.792865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729db301-ca87-40c8-be3c-13ff66cf294c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:49.861621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6df45-b112-4309-8eec-88332d3b2530 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:49.913337 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:49.924399 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260bdf7f-21ff-4f00-9d44-275e7ffb85cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:49.925055 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bdf7f-21ff-4f00-9d44-275e7ffb85cd None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:49.925338 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bdf7f-21ff-4f00-9d44-275e7ffb85cd None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:49.925612 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bdf7f-21ff-4f00-9d44-275e7ffb85cd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:49.945708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260bdf7f-21ff-4f00-9d44-275e7ffb85cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:49.951619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-260bdf7f-21ff-4f00-9d44-275e7ffb85cd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:19:49.951619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:49.951619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:49.951619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:49.951619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:49.951619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:49.951619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:49.951619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:49.951619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:49.951619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:49.951619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:49.951619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:49.951619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:49.951619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:49.951619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:49.951619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:49.952510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:49.952510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:49.952510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:49.952510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:49.952510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:49.952510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:49.952510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:49.952510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:49.952510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:49.952510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:49.952510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:49.952510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:49.952510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:49.952510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:49.952510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:49.952510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:49.953301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:49.953301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:49.953301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:49.953301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:49.953301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:49.953301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:49.953301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:49.953301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:49.953301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:49.953301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:49.953301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:19:49.953301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:19:49.955021 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 117/225] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:19:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:19:49.960834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260bdf7f-21ff-4f00-9d44-275e7ffb85cd None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:49.967806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729db301-ca87-40c8-be3c-13ff66cf294c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:50.038629 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec257d5c-413a-4f41-8c66-e5ca769dcd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:50.039213 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec257d5c-413a-4f41-8c66-e5ca769dcd53 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:50.039497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec257d5c-413a-4f41-8c66-e5ca769dcd53 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:50.039775 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec257d5c-413a-4f41-8c66-e5ca769dcd53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:50.068065 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec257d5c-413a-4f41-8c66-e5ca769dcd53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:50.074018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-ec257d5c-413a-4f41-8c66-e5ca769dcd53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:19:50.074018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:50.074018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:50.074018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:50.074018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:50.074018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:50.074018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:50.074018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:50.074018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:50.074018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:50.074018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:50.074018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:50.074018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:50.074018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:50.074018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:50.074018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:50.075252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:50.075252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:50.075252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:50.075252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:50.075252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:50.075252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:50.075252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:50.075252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:50.075252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:50.075252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:50.075252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:50.075252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:50.075252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:50.075252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:50.075252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:50.075252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:50.076691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:50.076691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:50.076691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:50.076691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:50.076691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:50.076691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:50.076691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:50.076691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:50.076691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:50.076691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:50.076691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:19:50.076691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:19:50.076691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 109/226] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:19:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:19:50.087176 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec257d5c-413a-4f41-8c66-e5ca769dcd53 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:50.106272 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:50.161371 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260bdf7f-21ff-4f00-9d44-275e7ffb85cd None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:50.222384 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8afd3d-1d6a-49e6-96e4-6b167203a703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:50.223494 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8afd3d-1d6a-49e6-96e4-6b167203a703 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:50.223978 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8afd3d-1d6a-49e6-96e4-6b167203a703 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:50.224458 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8afd3d-1d6a-49e6-96e4-6b167203a703 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:50.225543 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8afd3d-1d6a-49e6-96e4-6b167203a703 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:50.226174 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8afd3d-1d6a-49e6-96e4-6b167203a703 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:50.227824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8afd3d-1d6a-49e6-96e4-6b167203a703 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:50.228809 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec257d5c-413a-4f41-8c66-e5ca769dcd53 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:50.248758 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 118/227] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:19:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:50.255190 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8afd3d-1d6a-49e6-96e4-6b167203a703 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:50.275142 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:50.285249 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f204f69-df01-49b7-8223-29ad17f0e99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:50.285920 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f204f69-df01-49b7-8223-29ad17f0e99b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:50.286217 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f204f69-df01-49b7-8223-29ad17f0e99b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:50.286503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f204f69-df01-49b7-8223-29ad17f0e99b None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:50.287265 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f204f69-df01-49b7-8223-29ad17f0e99b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:50.287582 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f204f69-df01-49b7-8223-29ad17f0e99b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:50.289206 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f204f69-df01-49b7-8223-29ad17f0e99b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:50.309289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 110/228] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:50.315532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f204f69-df01-49b7-8223-29ad17f0e99b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:50.460741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8afd3d-1d6a-49e6-96e4-6b167203a703 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:50.489042 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f204f69-df01-49b7-8223-29ad17f0e99b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:50.514789 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:50.525976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c581140-d6ab-4ccc-8b48-2fba4d570e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:50.526632 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c581140-d6ab-4ccc-8b48-2fba4d570e6e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:50.526913 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c581140-d6ab-4ccc-8b48-2fba4d570e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:50.527184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c581140-d6ab-4ccc-8b48-2fba4d570e6e None admin] PATH_INFO: `/v3/users/168e395a154c44fd826effd7718122ac` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:50.528463 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c581140-d6ab-4ccc-8b48-2fba4d570e6e None admin] RBAC: Authorizing `identity:get_user(user_id=168e395a154c44fd826effd7718122ac)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:50.531262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c581140-d6ab-4ccc-8b48-2fba4d570e6e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:50.533929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 119/229] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:50 2022] GET /identity/v3/users/168e395a154c44fd826effd7718122ac => generated 283 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:50.538632 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c581140-d6ab-4ccc-8b48-2fba4d570e6e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:50.555800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad381f-f048-4d32-8191-ae337228bf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:50.556426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad381f-f048-4d32-8191-ae337228bf77 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:50.556718 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad381f-f048-4d32-8191-ae337228bf77 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:50.556999 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad381f-f048-4d32-8191-ae337228bf77 None admin] PATH_INFO: `/v3/users/168e395a154c44fd826effd7718122ac` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:50.559555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad381f-f048-4d32-8191-ae337228bf77 None admin] RBAC: Authorizing `identity:get_user(user_id=168e395a154c44fd826effd7718122ac)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:50.561032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad381f-f048-4d32-8191-ae337228bf77 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:50.562701 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 111/230] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:50 2022] GET /identity/v3/users/168e395a154c44fd826effd7718122ac => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:50.568352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad381f-f048-4d32-8191-ae337228bf77 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:50.690304 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:50.729877 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c581140-d6ab-4ccc-8b48-2fba4d570e6e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:50.734568 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad381f-f048-4d32-8191-ae337228bf77 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:50.774386 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc97bd3-7250-4acb-872f-1043ddf823be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:50.775398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30becb-144b-435a-ae95-7d9dcafe8144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:50.775712 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc97bd3-7250-4acb-872f-1043ddf823be None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:50.776090 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30becb-144b-435a-ae95-7d9dcafe8144 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:50.776373 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30becb-144b-435a-ae95-7d9dcafe8144 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:50.776649 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30becb-144b-435a-ae95-7d9dcafe8144 None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:50.776866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc97bd3-7250-4acb-872f-1043ddf823be None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:50.777436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc97bd3-7250-4acb-872f-1043ddf823be None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:50.777836 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30becb-144b-435a-ae95-7d9dcafe8144 None admin] RBAC: Authorizing `identity:get_project(project_id=d659ef9d8a7e447cb15feaa6be1b2787)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:50.779071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc97bd3-7250-4acb-872f-1043ddf823be None admin] RBAC: Authorizing `identity:get_project(project_id=d659ef9d8a7e447cb15feaa6be1b2787)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:50.779381 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30becb-144b-435a-ae95-7d9dcafe8144 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:50.780944 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc97bd3-7250-4acb-872f-1043ddf823be None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:50.781407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 112/231] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:50 2022] GET /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:50.783356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 120/232] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:50 2022] GET /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:50.786806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66647b4-66ca-4a04-add5-bad50d5a3a7c None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:50.787351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66647b4-66ca-4a04-add5-bad50d5a3a7c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:50.787699 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc97bd3-7250-4acb-872f-1043ddf823be None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:50.788161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66647b4-66ca-4a04-add5-bad50d5a3a7c None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:50.789707 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 113/233] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:50.790644 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66647b4-66ca-4a04-add5-bad50d5a3a7c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:50.867069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:50.907577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc97bd3-7250-4acb-872f-1043ddf823be None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:50.920719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66647b4-66ca-4a04-add5-bad50d5a3a7c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:50.943428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b59aa-29d1-45b5-9c82-adee8ed86a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:50.944102 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b59aa-29d1-45b5-9c82-adee8ed86a26 None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:50.944389 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b59aa-29d1-45b5-9c82-adee8ed86a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:50.944914 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b59aa-29d1-45b5-9c82-adee8ed86a26 None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787/users/168e395a154c44fd826effd7718122ac/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:50.946939 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b59aa-29d1-45b5-9c82-adee8ed86a26 None admin] RBAC: Authorizing `identity:create_grant(project_id=d659ef9d8a7e447cb15feaa6be1b2787, user_id=168e395a154c44fd826effd7718122ac, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:50.948414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b59aa-29d1-45b5-9c82-adee8ed86a26 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:50.952142 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:50.959306 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a540e-c2e3-423c-a41f-1b82bd4bb6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:50.959984 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a540e-c2e3-423c-a41f-1b82bd4bb6e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:50.960270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a540e-c2e3-423c-a41f-1b82bd4bb6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:50.960547 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a540e-c2e3-423c-a41f-1b82bd4bb6e1 None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787/users/168e395a154c44fd826effd7718122ac/roles/73e29e4f51e6469facf94aaa698c5cfe` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:50.962560 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 121/234] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:19:50 2022] PUT /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787/users/168e395a154c44fd826effd7718122ac/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:50.962794 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a540e-c2e3-423c-a41f-1b82bd4bb6e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=d659ef9d8a7e447cb15feaa6be1b2787, user_id=168e395a154c44fd826effd7718122ac, role_id=73e29e4f51e6469facf94aaa698c5cfe)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:50.964298 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a540e-c2e3-423c-a41f-1b82bd4bb6e1 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:50.971465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a5dff-48bb-4c56-b96e-155dc7b30670 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:50.974455 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a5dff-48bb-4c56-b96e-155dc7b30670 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:50.975644 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a5dff-48bb-4c56-b96e-155dc7b30670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:50.980102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 114/235] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:19:50 2022] PUT /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787/users/168e395a154c44fd826effd7718122ac/roles/73e29e4f51e6469facf94aaa698c5cfe => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:50.986084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417774f-80ac-4e8e-9311-e29fba84b1fb None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:50.986084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417774f-80ac-4e8e-9311-e29fba84b1fb None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:50.986084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417774f-80ac-4e8e-9311-e29fba84b1fb None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:50.986084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 115/236] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:51.009562 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea10ef7-1180-42b1-8c19-4f82c5d97e4f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.010123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea10ef7-1180-42b1-8c19-4f82c5d97e4f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.010580 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea10ef7-1180-42b1-8c19-4f82c5d97e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.012321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d44a5dff-48bb-4c56-b96e-155dc7b30670 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:51.021924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d44a5dff-48bb-4c56-b96e-155dc7b30670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:51.051019 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6ea10ef7-1180-42b1-8c19-4f82c5d97e4f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:51.052565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6ea10ef7-1180-42b1-8c19-4f82c5d97e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:51.112229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 122/237] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:51.143773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a5dff-48bb-4c56-b96e-155dc7b30670 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:51.195431 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 116/238] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:51.214886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:51.220009 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea10ef7-1180-42b1-8c19-4f82c5d97e4f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:51.279473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a5dff-48bb-4c56-b96e-155dc7b30670 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:51.326145 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54d111-5b00-4cb1-8cee-6d58528dcfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:51.327726 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54d111-5b00-4cb1-8cee-6d58528dcfb9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.328367 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54d111-5b00-4cb1-8cee-6d58528dcfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.328936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54d111-5b00-4cb1-8cee-6d58528dcfb9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.331671 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a54d111-5b00-4cb1-8cee-6d58528dcfb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:51.332227 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a54d111-5b00-4cb1-8cee-6d58528dcfb9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:51.334307 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 123/239] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:19:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:51.335287 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2b992-8f8d-466b-b315-6c8ea0438097 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.335572 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2b992-8f8d-466b-b315-6c8ea0438097 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.335840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2b992-8f8d-466b-b315-6c8ea0438097 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.337080 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 124/240] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:51.343090 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594debd3-abed-4432-993c-2c7d9a3c1b23 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.343383 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594debd3-abed-4432-993c-2c7d9a3c1b23 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.343660 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594debd3-abed-4432-993c-2c7d9a3c1b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.347749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea10ef7-1180-42b1-8c19-4f82c5d97e4f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:51.363213 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-594debd3-abed-4432-993c-2c7d9a3c1b23 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:51.364410 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-594debd3-abed-4432-993c-2c7d9a3c1b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:51.374592 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:51.397674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990d9c5-fc32-43e1-a94f-d905ae5025d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:51.398684 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990d9c5-fc32-43e1-a94f-d905ae5025d8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.398974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990d9c5-fc32-43e1-a94f-d905ae5025d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.399522 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990d9c5-fc32-43e1-a94f-d905ae5025d8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.414461 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990d9c5-fc32-43e1-a94f-d905ae5025d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:51.422353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-6990d9c5-fc32-43e1-a94f-d905ae5025d8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:19:51.422353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:51.422353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:51.422353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:51.422353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:51.422353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:51.422353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:51.422353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:51.422353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:51.422353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:51.422353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:51.422353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:51.422353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:51.422353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:51.422353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:51.422353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:51.423281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:51.423281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:51.423281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:51.423281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:51.423281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:51.423281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:51.423281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:51.423281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:51.423281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:51.423281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:51.423281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:51.423281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:51.423281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:51.423281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:51.423281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:51.423281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:51.424096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:51.424096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:51.424096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:51.424096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:51.424096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:51.424096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:51.424096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:51.424096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:51.424096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:51.424096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:51.424096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:19:51.424096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:19:51.424096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 117/241] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:19:51.424954 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990d9c5-fc32-43e1-a94f-d905ae5025d8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:51.442997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 125/242] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:19:51.451046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594debd3-abed-4432-993c-2c7d9a3c1b23 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:51.525837 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:51.552077 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990d9c5-fc32-43e1-a94f-d905ae5025d8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:51.590786 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594debd3-abed-4432-993c-2c7d9a3c1b23 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:51.601665 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8913f-192c-4a7a-b268-334b4ea7333d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:51.602741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8913f-192c-4a7a-b268-334b4ea7333d None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.603189 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8913f-192c-4a7a-b268-334b4ea7333d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.603686 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8913f-192c-4a7a-b268-334b4ea7333d None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.604539 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8913f-192c-4a7a-b268-334b4ea7333d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:51.606847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8913f-192c-4a7a-b268-334b4ea7333d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:51.629579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025b359-fa2b-4ca7-88bd-01069fe4d2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:51.630394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025b359-fa2b-4ca7-88bd-01069fe4d2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.630683 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025b359-fa2b-4ca7-88bd-01069fe4d2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.630961 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025b359-fa2b-4ca7-88bd-01069fe4d2c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.631695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025b359-fa2b-4ca7-88bd-01069fe4d2c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:51.632104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025b359-fa2b-4ca7-88bd-01069fe4d2c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:51.633635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025b359-fa2b-4ca7-88bd-01069fe4d2c7 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:51.635878 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 118/243] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Aug 1 15:19:51 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:51.637039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8913f-192c-4a7a-b268-334b4ea7333d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:51.659564 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 126/244] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:19:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:51.660586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025b359-fa2b-4ca7-88bd-01069fe4d2c7 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:51.680751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:51.766315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8913f-192c-4a7a-b268-334b4ea7333d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:51.807837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d9c8f-342a-405d-9180-1ed9885d7786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:51.808858 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d9c8f-342a-405d-9180-1ed9885d7786 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.809212 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d9c8f-342a-405d-9180-1ed9885d7786 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.809583 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d9c8f-342a-405d-9180-1ed9885d7786 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.823302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d9c8f-342a-405d-9180-1ed9885d7786 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:51.829690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-1d1d9c8f-342a-405d-9180-1ed9885d7786 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:19:51.829690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:51.829690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:51.829690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:51.829690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:51.829690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:51.829690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:51.829690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:51.829690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:51.829690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:51.829690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:51.829690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:51.829690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:51.829690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:51.829690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:51.829690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:51.830682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:51.830682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:51.830682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:51.830682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:51.830682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:51.830682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:51.830682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:51.830682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:51.830682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:51.830682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:51.830682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:51.830682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:51.830682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:51.830682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:51.830682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:51.830682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:51.831489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:51.831489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:51.831489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:51.831489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:51.831489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:51.831489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:51.831489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:51.831489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:51.831489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:51.831489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:51.831489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:19:51.831489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:19:51.832102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 119/245] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:51.843142 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d9c8f-342a-405d-9180-1ed9885d7786 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:51.889476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025b359-fa2b-4ca7-88bd-01069fe4d2c7 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:51.910964 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:51.934011 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef98deb-3a14-456b-a459-a1f5c79269fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:51.934901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef98deb-3a14-456b-a459-a1f5c79269fb None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.935357 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef98deb-3a14-456b-a459-a1f5c79269fb None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.935777 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef98deb-3a14-456b-a459-a1f5c79269fb None admin] PATH_INFO: `/v3/users/51325609aace494998b99ac3bce82c44` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.952161 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef98deb-3a14-456b-a459-a1f5c79269fb None admin] RBAC: Authorizing `identity:get_user(user_id=51325609aace494998b99ac3bce82c44)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:51.954007 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef98deb-3a14-456b-a459-a1f5c79269fb None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:51.957206 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 127/246] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:51 2022] GET /identity/v3/users/51325609aace494998b99ac3bce82c44 => generated 297 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:51.966019 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef98deb-3a14-456b-a459-a1f5c79269fb None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:51.977443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:51.983339 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d9c8f-342a-405d-9180-1ed9885d7786 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:52.019585 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560a5b6-1695-4d88-afd7-e88006ce58d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:52.020302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560a5b6-1695-4d88-afd7-e88006ce58d1 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.020589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560a5b6-1695-4d88-afd7-e88006ce58d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.020903 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560a5b6-1695-4d88-afd7-e88006ce58d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.021688 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e560a5b6-1695-4d88-afd7-e88006ce58d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:52.022023 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e560a5b6-1695-4d88-afd7-e88006ce58d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:52.023417 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e560a5b6-1695-4d88-afd7-e88006ce58d1 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:52.032508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 120/247] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:19:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:52.038063 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560a5b6-1695-4d88-afd7-e88006ce58d1 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:52.087163 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef98deb-3a14-456b-a459-a1f5c79269fb None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:52.126366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb14419-2903-4c99-a172-216521f172f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:52.127324 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb14419-2903-4c99-a172-216521f172f4 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.127785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb14419-2903-4c99-a172-216521f172f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.128337 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb14419-2903-4c99-a172-216521f172f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.153591 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560a5b6-1695-4d88-afd7-e88006ce58d1 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:52.179120 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:52.194066 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97ff5d-1c1f-4a27-8b06-a5e93fe64bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:52.195029 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97ff5d-1c1f-4a27-8b06-a5e93fe64bbf None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.195240 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97ff5d-1c1f-4a27-8b06-a5e93fe64bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.195477 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97ff5d-1c1f-4a27-8b06-a5e93fe64bbf None admin] PATH_INFO: `/v3/users/9d56df6133fd4e0d83b6090aa2abdcc5` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.196694 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97ff5d-1c1f-4a27-8b06-a5e93fe64bbf None admin] RBAC: Authorizing `identity:get_user(user_id=9d56df6133fd4e0d83b6090aa2abdcc5)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:52.199082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97ff5d-1c1f-4a27-8b06-a5e93fe64bbf None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:52.200789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 121/248] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:52 2022] GET /identity/v3/users/9d56df6133fd4e0d83b6090aa2abdcc5 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:52.206515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97ff5d-1c1f-4a27-8b06-a5e93fe64bbf None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:52.223656 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb14419-2903-4c99-a172-216521f172f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:52.225329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb14419-2903-4c99-a172-216521f172f4 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:52.242180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:52.346803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97ff5d-1c1f-4a27-8b06-a5e93fe64bbf None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:52.356972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 128/249] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:19:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:19:52.362791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb14419-2903-4c99-a172-216521f172f4 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:52.388136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55383177-4920-4376-b537-c9743d6f6a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:52.388924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55383177-4920-4376-b537-c9743d6f6a22 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.389205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55383177-4920-4376-b537-c9743d6f6a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.389488 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55383177-4920-4376-b537-c9743d6f6a22 None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.391544 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55383177-4920-4376-b537-c9743d6f6a22 None admin] RBAC: Authorizing `identity:get_project(project_id=d659ef9d8a7e447cb15feaa6be1b2787)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:52.393046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55383177-4920-4376-b537-c9743d6f6a22 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:52.394980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 122/250] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:52 2022] GET /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:52.401852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55383177-4920-4376-b537-c9743d6f6a22 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:52.467643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:52.501969 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb14419-2903-4c99-a172-216521f172f4 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:52.525807 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:52.535564 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55383177-4920-4376-b537-c9743d6f6a22 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:52.551664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b11578-d751-4936-8b32-7b67201ba659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:52.552421 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b11578-d751-4936-8b32-7b67201ba659 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.553050 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b11578-d751-4936-8b32-7b67201ba659 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.553400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b11578-d751-4936-8b32-7b67201ba659 None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.555816 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b11578-d751-4936-8b32-7b67201ba659 None admin] RBAC: Authorizing `identity:get_project(project_id=d659ef9d8a7e447cb15feaa6be1b2787)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:52.557161 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b11578-d751-4936-8b32-7b67201ba659 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:52.561046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 129/251] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:52 2022] GET /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:52.566003 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b11578-d751-4936-8b32-7b67201ba659 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:52.585352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e96ad-b162-4b58-b222-a111792d4809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:52.586437 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e96ad-b162-4b58-b222-a111792d4809 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.586945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e96ad-b162-4b58-b222-a111792d4809 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.587618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e96ad-b162-4b58-b222-a111792d4809 None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787/users/9d56df6133fd4e0d83b6090aa2abdcc5/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.590563 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e96ad-b162-4b58-b222-a111792d4809 None admin] RBAC: Authorizing `identity:create_grant(project_id=d659ef9d8a7e447cb15feaa6be1b2787, user_id=9d56df6133fd4e0d83b6090aa2abdcc5, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:52.592838 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e96ad-b162-4b58-b222-a111792d4809 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:52.612910 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 123/252] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:19:52 2022] PUT /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787/users/9d56df6133fd4e0d83b6090aa2abdcc5/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:52.688688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b11578-d751-4936-8b32-7b67201ba659 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:52.738818 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458727d-cc62-4560-bf2e-b90139d82496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:52.739373 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458727d-cc62-4560-bf2e-b90139d82496 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.739655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458727d-cc62-4560-bf2e-b90139d82496 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.739928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458727d-cc62-4560-bf2e-b90139d82496 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.740730 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458727d-cc62-4560-bf2e-b90139d82496 None admin] RBAC: Adding query filter params (role.id=fdbe7b923fed4101826cfb93a92e3fbf, scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787, user.id=51325609aace494998b99ac3bce82c44) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:52.741043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458727d-cc62-4560-bf2e-b90139d82496 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:52.742553 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458727d-cc62-4560-bf2e-b90139d82496 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:52.755098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 130/253] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:19:52 2022] GET /identity/v3/role_assignments?user.id=51325609aace494998b99ac3bce82c44&scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787&role.id=fdbe7b923fed4101826cfb93a92e3fbf => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:53.461231 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11f8d7-bff0-435f-b630-020e02127ef2 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.461952 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11f8d7-bff0-435f-b630-020e02127ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.462428 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11f8d7-bff0-435f-b630-020e02127ef2 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.464238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 124/254] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:53.469823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68bcef-d918-4fca-a37f-224d64c979f0 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.470738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68bcef-d918-4fca-a37f-224d64c979f0 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.471223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68bcef-d918-4fca-a37f-224d64c979f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.486515 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:53.493783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-5c68bcef-d918-4fca-a37f-224d64c979f0 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:53.495855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5c68bcef-d918-4fca-a37f-224d64c979f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:53.575002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 131/255] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:19:53.615860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11f8d7-bff0-435f-b630-020e02127ef2 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:53.657271 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec5d5d-17d4-4437-b4ce-439bc7c7c776 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.657880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec5d5d-17d4-4437-b4ce-439bc7c7c776 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.658377 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec5d5d-17d4-4437-b4ce-439bc7c7c776 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.659918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 132/256] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:19:53.665072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a8980-98af-461e-af84-42d030a5028a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.665633 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a8980-98af-461e-af84-42d030a5028a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.666133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a8980-98af-461e-af84-42d030a5028a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.710900 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1e0a8980-98af-461e-af84-42d030a5028a None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:53.712488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1e0a8980-98af-461e-af84-42d030a5028a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:53.752204 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11f8d7-bff0-435f-b630-020e02127ef2 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:53.810339 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629133fb-e135-44c3-a931-168efd698688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:53.811004 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629133fb-e135-44c3-a931-168efd698688 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.811289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629133fb-e135-44c3-a931-168efd698688 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.811566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629133fb-e135-44c3-a931-168efd698688 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.831523 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629133fb-e135-44c3-a931-168efd698688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:53.837084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-629133fb-e135-44c3-a931-168efd698688 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:19:53.837084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:53.837084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:53.837084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:53.837084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:53.837084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:53.837084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:53.837084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:53.837084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:53.837084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:53.837084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:53.837084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:53.837084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:53.837084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:53.837084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:53.837084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:53.837966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:53.837966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:53.837966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:53.837966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:53.837966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:53.837966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:53.837966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:53.837966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:53.837966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:53.837966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:53.837966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:53.837966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:53.837966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:53.837966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:53.837966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:53.837966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:53.838798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:53.838798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:53.838798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:53.838798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:53.838798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:53.838798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:53.838798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:53.838798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:53.838798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:53.838798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:53.838798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:19:53.838798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:19:53.838798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 125/257] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:19:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:53.864623 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629133fb-e135-44c3-a931-168efd698688 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:53.886969 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 133/258] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:19:53.902312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:53.922015 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a8980-98af-461e-af84-42d030a5028a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:54.038739 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:54.093022 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a8980-98af-461e-af84-42d030a5028a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:54.104598 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629133fb-e135-44c3-a931-168efd698688 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:54.133751 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f945f1a-36f0-4557-8f63-f1096fb989ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:54.134790 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f945f1a-36f0-4557-8f63-f1096fb989ed None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:54.135304 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f945f1a-36f0-4557-8f63-f1096fb989ed None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:54.136540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f945f1a-36f0-4557-8f63-f1096fb989ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:54.156790 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f945f1a-36f0-4557-8f63-f1096fb989ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:54.162354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-0f945f1a-36f0-4557-8f63-f1096fb989ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:19:54.162354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:54.162354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:54.162354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:54.162354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:54.162354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:54.162354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:54.162354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:54.162354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:54.162354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:54.162354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:54.162354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:54.162354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:54.162354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:54.162354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:54.162354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:54.163215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:54.163215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:54.163215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:54.163215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:54.163215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:54.163215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:54.163215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:54.163215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:54.163215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:54.163215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:54.163215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:54.163215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:54.163215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:54.163215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:54.163215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:54.163215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:54.164006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:54.164006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:54.164006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:54.164006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:54.164006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:54.164006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:54.164006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:54.164006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:54.164006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:54.164006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:54.164006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:19:54.164006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:19:54.172478 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 134/259] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:19:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:19:54.183008 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f945f1a-36f0-4557-8f63-f1096fb989ed None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:54.185636 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89414af-d0ac-4d8e-8502-dde2031221d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:54.193939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89414af-d0ac-4d8e-8502-dde2031221d3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:54.193939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89414af-d0ac-4d8e-8502-dde2031221d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:54.193939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89414af-d0ac-4d8e-8502-dde2031221d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:54.193939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89414af-d0ac-4d8e-8502-dde2031221d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:54.193939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89414af-d0ac-4d8e-8502-dde2031221d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:54.193939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89414af-d0ac-4d8e-8502-dde2031221d3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:54.233917 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 126/260] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:54.234863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89414af-d0ac-4d8e-8502-dde2031221d3 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:54.430026 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f945f1a-36f0-4557-8f63-f1096fb989ed None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:54.445595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:54.483487 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:54.494197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89414af-d0ac-4d8e-8502-dde2031221d3 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:54.504406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78faff-bed1-47d1-8f79-8f3a91acde35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:54.505318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78faff-bed1-47d1-8f79-8f3a91acde35 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:54.505789 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78faff-bed1-47d1-8f79-8f3a91acde35 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:54.506301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78faff-bed1-47d1-8f79-8f3a91acde35 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:54.507271 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78faff-bed1-47d1-8f79-8f3a91acde35 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:54.507791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78faff-bed1-47d1-8f79-8f3a91acde35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:54.509359 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78faff-bed1-47d1-8f79-8f3a91acde35 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:54.523916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 135/261] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:19:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:54.530640 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78faff-bed1-47d1-8f79-8f3a91acde35 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:54.573208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632d2eb-66c8-4feb-af49-1c70e6f14d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:54.574241 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632d2eb-66c8-4feb-af49-1c70e6f14d78 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:54.574734 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632d2eb-66c8-4feb-af49-1c70e6f14d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:54.575201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632d2eb-66c8-4feb-af49-1c70e6f14d78 None admin] PATH_INFO: `/v3/users/168e395a154c44fd826effd7718122ac` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:54.576610 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632d2eb-66c8-4feb-af49-1c70e6f14d78 None admin] RBAC: Authorizing `identity:get_user(user_id=168e395a154c44fd826effd7718122ac)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:54.578389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632d2eb-66c8-4feb-af49-1c70e6f14d78 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:54.581090 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 127/262] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:54 2022] GET /identity/v3/users/168e395a154c44fd826effd7718122ac => generated 283 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:54.586724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632d2eb-66c8-4feb-af49-1c70e6f14d78 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:54.751519 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78faff-bed1-47d1-8f79-8f3a91acde35 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:54.785297 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632d2eb-66c8-4feb-af49-1c70e6f14d78 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:54.825911 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d211f-6e26-4729-90e9-d1172719ed58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:54.826927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d211f-6e26-4729-90e9-d1172719ed58 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:54.827401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d211f-6e26-4729-90e9-d1172719ed58 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:54.827864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d211f-6e26-4729-90e9-d1172719ed58 None admin] PATH_INFO: `/v3/users/168e395a154c44fd826effd7718122ac` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:54.829408 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d211f-6e26-4729-90e9-d1172719ed58 None admin] RBAC: Authorizing `identity:get_user(user_id=168e395a154c44fd826effd7718122ac)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:54.831158 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d211f-6e26-4729-90e9-d1172719ed58 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:54.835338 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 136/263] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:54 2022] GET /identity/v3/users/168e395a154c44fd826effd7718122ac => generated 283 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:54.841101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d211f-6e26-4729-90e9-d1172719ed58 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:54.869179 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8017a3-708c-4736-8fad-3eeae05e2c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:54.870821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8017a3-708c-4736-8fad-3eeae05e2c30 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:54.871295 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8017a3-708c-4736-8fad-3eeae05e2c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:54.871759 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8017a3-708c-4736-8fad-3eeae05e2c30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:54.910254 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:54.963379 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:55.002724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8017a3-708c-4736-8fad-3eeae05e2c30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:55.004293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8017a3-708c-4736-8fad-3eeae05e2c30 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:55.012940 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d211f-6e26-4729-90e9-d1172719ed58 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:55.068795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894df13-5466-49db-8ee6-059ed2754cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:55.069970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894df13-5466-49db-8ee6-059ed2754cfe None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:55.070391 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894df13-5466-49db-8ee6-059ed2754cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:55.070820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894df13-5466-49db-8ee6-059ed2754cfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:55.146108 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894df13-5466-49db-8ee6-059ed2754cfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:55.147850 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894df13-5466-49db-8ee6-059ed2754cfe None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:55.155469 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 128/264] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:19:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 15:19:55.175771 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8017a3-708c-4736-8fad-3eeae05e2c30 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:55.212784 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:55.251863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:55.261980 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 137/265] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:19:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:19:55.267524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894df13-5466-49db-8ee6-059ed2754cfe None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:55.345442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8017a3-708c-4736-8fad-3eeae05e2c30 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:55.410968 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2633c8c-f063-4da1-a214-2a4b995ecd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:55.411995 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2633c8c-f063-4da1-a214-2a4b995ecd60 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:55.412381 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2633c8c-f063-4da1-a214-2a4b995ecd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:55.412769 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2633c8c-f063-4da1-a214-2a4b995ecd60 None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:55.414575 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2633c8c-f063-4da1-a214-2a4b995ecd60 None admin] RBAC: Authorizing `identity:get_project(project_id=d659ef9d8a7e447cb15feaa6be1b2787)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:55.417334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2633c8c-f063-4da1-a214-2a4b995ecd60 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:55.419346 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 129/266] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:55 2022] GET /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:55.426247 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2633c8c-f063-4da1-a214-2a4b995ecd60 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:55.489402 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894df13-5466-49db-8ee6-059ed2754cfe None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:55.549938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cec38f-1b5a-45d3-a2c6-6536ef6905f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:55.550510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cec38f-1b5a-45d3-a2c6-6536ef6905f8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:55.550792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cec38f-1b5a-45d3-a2c6-6536ef6905f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:55.551066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cec38f-1b5a-45d3-a2c6-6536ef6905f8 None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:55.552345 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cec38f-1b5a-45d3-a2c6-6536ef6905f8 None admin] RBAC: Authorizing `identity:get_project(project_id=d659ef9d8a7e447cb15feaa6be1b2787)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:55.553800 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cec38f-1b5a-45d3-a2c6-6536ef6905f8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:55.560669 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 138/267] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:55 2022] GET /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787 => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:55.562015 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4227960-83a3-4723-883d-550bca22be83 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:55.562319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4227960-83a3-4723-883d-550bca22be83 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:55.562562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4227960-83a3-4723-883d-550bca22be83 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:55.564013 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 139/268] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:55.572238 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4227960-83a3-4723-883d-550bca22be83 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:55.605178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:55.618404 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2633c8c-f063-4da1-a214-2a4b995ecd60 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:55.634563 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:55.667205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9153f-99a0-406b-8a00-a4452e4a236b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:55.667976 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9153f-99a0-406b-8a00-a4452e4a236b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:55.668261 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9153f-99a0-406b-8a00-a4452e4a236b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:55.668532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9153f-99a0-406b-8a00-a4452e4a236b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:55.669336 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9153f-99a0-406b-8a00-a4452e4a236b None admin] RBAC: Adding query filter params (role.id=7bdfa43dc1d046f7965ffe52b74bc58f, scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787, user.id=168e395a154c44fd826effd7718122ac) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:55.669649 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9153f-99a0-406b-8a00-a4452e4a236b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:55.671161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9153f-99a0-406b-8a00-a4452e4a236b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:55.677425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 130/269] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:19:55 2022] GET /identity/v3/role_assignments?user.id=168e395a154c44fd826effd7718122ac&scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787&role.id=7bdfa43dc1d046f7965ffe52b74bc58f => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:55.678861 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c0eea-126e-4815-9d42-e76dd4cb22ae None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:55.679109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c0eea-126e-4815-9d42-e76dd4cb22ae None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:55.679397 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c0eea-126e-4815-9d42-e76dd4cb22ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:55.707512 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-136c0eea-126e-4815-9d42-e76dd4cb22ae None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:55.709731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-136c0eea-126e-4815-9d42-e76dd4cb22ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:55.713617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4227960-83a3-4723-883d-550bca22be83 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:55.766999 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47355bfc-dc53-47ff-a66f-e2223c360bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:55.768104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47355bfc-dc53-47ff-a66f-e2223c360bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:55.768612 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47355bfc-dc53-47ff-a66f-e2223c360bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:55.769085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47355bfc-dc53-47ff-a66f-e2223c360bb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:55.770170 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47355bfc-dc53-47ff-a66f-e2223c360bb1 None admin] RBAC: Adding query filter params (role.id=73e29e4f51e6469facf94aaa698c5cfe, scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787, user.id=168e395a154c44fd826effd7718122ac) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:55.770752 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47355bfc-dc53-47ff-a66f-e2223c360bb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:55.772679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47355bfc-dc53-47ff-a66f-e2223c360bb1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:55.781149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 140/270] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:19:55 2022] GET /identity/v3/role_assignments?user.id=168e395a154c44fd826effd7718122ac&scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787&role.id=73e29e4f51e6469facf94aaa698c5cfe => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:55.782843 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad5acd-e5d3-4fea-aeb4-738bcd74681f None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:55.783332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad5acd-e5d3-4fea-aeb4-738bcd74681f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:55.783803 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad5acd-e5d3-4fea-aeb4-738bcd74681f None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:55.785988 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 141/271] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:55.787773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86fb25-ba7a-4719-bf6e-f7e1aed01f39 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:55.788311 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86fb25-ba7a-4719-bf6e-f7e1aed01f39 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:55.788825 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86fb25-ba7a-4719-bf6e-f7e1aed01f39 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:55.790631 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 142/272] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:55.796058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff3182-d5e5-4d6b-b84b-5f5e686fca9c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:55.796823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff3182-d5e5-4d6b-b84b-5f5e686fca9c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:55.797332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff3182-d5e5-4d6b-b84b-5f5e686fca9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:55.807740 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 131/273] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:55.810139 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c4c18-24f3-430a-9891-b7e11f6e0f95 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:55.810577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c4c18-24f3-430a-9891-b7e11f6e0f95 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:55.811049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c4c18-24f3-430a-9891-b7e11f6e0f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:55.835318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e8ff3182-d5e5-4d6b-b84b-5f5e686fca9c None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:55.836767 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e8ff3182-d5e5-4d6b-b84b-5f5e686fca9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:55.852477 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-245c4c18-24f3-430a-9891-b7e11f6e0f95 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:55.855586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-245c4c18-24f3-430a-9891-b7e11f6e0f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:55.949193 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 143/274] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:55.951217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff3182-d5e5-4d6b-b84b-5f5e686fca9c None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:55.967598 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 132/275] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:55.971526 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc6e06-b592-46d9-9ca6-5408cffd86ef None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:55.972053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc6e06-b592-46d9-9ca6-5408cffd86ef None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:55.972475 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc6e06-b592-46d9-9ca6-5408cffd86ef None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:55.974053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 133/276] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:55.974946 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc6e06-b592-46d9-9ca6-5408cffd86ef None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:55.991093 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:56.029543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:56.087420 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff3182-d5e5-4d6b-b84b-5f5e686fca9c None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:56.111970 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc6e06-b592-46d9-9ca6-5408cffd86ef None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:56.122918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d67120-d778-406f-975b-9fcf210069e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:56.123566 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d67120-d778-406f-975b-9fcf210069e5 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:56.123848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d67120-d778-406f-975b-9fcf210069e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:56.124119 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d67120-d778-406f-975b-9fcf210069e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:56.125297 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d67120-d778-406f-975b-9fcf210069e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:56.126788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d67120-d778-406f-975b-9fcf210069e5 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:56.128410 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 144/277] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:19:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:56.129588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34525e-a930-403e-8d6e-2c929ee3a14c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:56.129875 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34525e-a930-403e-8d6e-2c929ee3a14c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:56.130184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34525e-a930-403e-8d6e-2c929ee3a14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:56.148957 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-dc34525e-a930-403e-8d6e-2c929ee3a14c None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:56.150170 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-dc34525e-a930-403e-8d6e-2c929ee3a14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:56.151696 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a946c-bd09-4da9-917a-2038d284b8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:56.152340 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a946c-bd09-4da9-917a-2038d284b8fb None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:56.152626 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a946c-bd09-4da9-917a-2038d284b8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:56.152901 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a946c-bd09-4da9-917a-2038d284b8fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:56.163139 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a946c-bd09-4da9-917a-2038d284b8fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:56.168439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-550a946c-bd09-4da9-917a-2038d284b8fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:19:56.168439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:56.168439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:56.168439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:56.168439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:56.168439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:56.168439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:56.168439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:56.168439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:56.168439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:56.168439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:56.168439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:56.168439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:56.168439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:56.168439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:56.168439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:56.169307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:56.169307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:56.169307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:56.169307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:56.169307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:56.169307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:56.169307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:56.169307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:56.169307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:56.169307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:56.169307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:56.169307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:56.169307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:56.169307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:56.169307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:56.169307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:56.170157 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:56.170157 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:56.170157 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:56.170157 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:56.170157 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:56.170157 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:56.170157 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:56.170157 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:56.170157 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:56.170157 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:56.170157 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:19:56.170157 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:19:56.170773 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 134/278] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:56.171231 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a946c-bd09-4da9-917a-2038d284b8fb None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:56.219973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 145/279] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:56.220797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34525e-a930-403e-8d6e-2c929ee3a14c None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:56.241033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:56.277089 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:56.283340 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a946c-bd09-4da9-917a-2038d284b8fb None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:56.317147 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78956a62-9293-4b21-8491-c4ad1d020229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:56.317789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78956a62-9293-4b21-8491-c4ad1d020229 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:56.318102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78956a62-9293-4b21-8491-c4ad1d020229 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:56.318419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78956a62-9293-4b21-8491-c4ad1d020229 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:56.323992 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34525e-a930-403e-8d6e-2c929ee3a14c None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:56.329374 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78956a62-9293-4b21-8491-c4ad1d020229 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:56.335293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-78956a62-9293-4b21-8491-c4ad1d020229 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:19:56.335293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:56.335293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:56.335293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:56.335293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:56.335293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:56.335293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:56.335293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:56.335293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:56.335293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:56.335293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:56.335293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:56.335293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:56.335293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:56.335293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:56.335293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:56.336182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:56.336182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:56.336182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:56.336182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:56.336182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:56.336182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:56.336182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:56.336182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:56.336182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:56.336182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:56.336182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:56.336182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:56.336182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:56.336182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:56.336182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:56.336182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:56.337046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:56.337046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:56.337046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:56.337046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:56.337046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:56.337046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:56.337046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:56.337046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:56.337046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:56.337046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:56.337046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:19:56.337046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:19:56.337046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 135/280] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:56.337744 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78956a62-9293-4b21-8491-c4ad1d020229 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:56.362785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67747215-0b6a-4c44-b0f7-07f3b5f21c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:56.363410 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67747215-0b6a-4c44-b0f7-07f3b5f21c4d None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:56.363715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67747215-0b6a-4c44-b0f7-07f3b5f21c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:56.364069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67747215-0b6a-4c44-b0f7-07f3b5f21c4d None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:56.364676 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67747215-0b6a-4c44-b0f7-07f3b5f21c4d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:56.366155 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67747215-0b6a-4c44-b0f7-07f3b5f21c4d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:56.388663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 146/281] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Aug 1 15:19:56 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:56.389489 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67747215-0b6a-4c44-b0f7-07f3b5f21c4d None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:56.460348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78956a62-9293-4b21-8491-c4ad1d020229 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:56.468598 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:56.501180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bd436-107c-459d-b6aa-ac82843dc69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:56.501825 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bd436-107c-459d-b6aa-ac82843dc69d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:56.502165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bd436-107c-459d-b6aa-ac82843dc69d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:56.502428 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bd436-107c-459d-b6aa-ac82843dc69d None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:56.503169 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608bd436-107c-459d-b6aa-ac82843dc69d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:56.503480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608bd436-107c-459d-b6aa-ac82843dc69d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:56.504909 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608bd436-107c-459d-b6aa-ac82843dc69d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:56.508404 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67747215-0b6a-4c44-b0f7-07f3b5f21c4d None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:56.515067 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 136/282] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:19:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:56.515752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bd436-107c-459d-b6aa-ac82843dc69d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:56.540652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:56.554571 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf5397-6e0e-41c0-9ab8-72e1d05a3965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:56.555223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf5397-6e0e-41c0-9ab8-72e1d05a3965 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:56.555527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf5397-6e0e-41c0-9ab8-72e1d05a3965 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:56.555797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf5397-6e0e-41c0-9ab8-72e1d05a3965 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:56.569827 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf5397-6e0e-41c0-9ab8-72e1d05a3965 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:56.576162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-bccf5397-6e0e-41c0-9ab8-72e1d05a3965 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:19:56.576162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:56.576162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:56.576162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:56.576162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:56.576162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:56.576162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:56.576162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:56.576162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:56.576162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:56.576162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:56.576162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:56.576162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:56.576162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:56.576162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:56.576162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:56.577139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:56.577139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:56.577139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:56.577139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:56.577139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:56.577139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:56.577139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:56.577139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:56.577139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:56.577139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:56.577139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:56.577139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:56.577139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:56.577139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:56.577139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:56.577139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:56.577979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:56.577979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:56.577979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:56.577979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:56.577979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:56.577979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:56.577979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:56.577979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:56.577979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:56.577979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:56.577979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:19:56.577979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:19:56.577979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 147/283] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:56.583056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf5397-6e0e-41c0-9ab8-72e1d05a3965 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:56.639601 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bd436-107c-459d-b6aa-ac82843dc69d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:56.689312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456d24f-e2f7-4d76-a3b9-0ae31480f63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:56.690019 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456d24f-e2f7-4d76-a3b9-0ae31480f63c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:56.690306 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456d24f-e2f7-4d76-a3b9-0ae31480f63c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:56.690615 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456d24f-e2f7-4d76-a3b9-0ae31480f63c None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:56.691469 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456d24f-e2f7-4d76-a3b9-0ae31480f63c None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:56.691782 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456d24f-e2f7-4d76-a3b9-0ae31480f63c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:56.693322 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456d24f-e2f7-4d76-a3b9-0ae31480f63c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:56.704270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 137/284] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:19:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:56.705144 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456d24f-e2f7-4d76-a3b9-0ae31480f63c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:56.716703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf5397-6e0e-41c0-9ab8-72e1d05a3965 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:56.749781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:56.754785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b8b7e-3582-4be2-a35f-27a608bad4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:56.755555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b8b7e-3582-4be2-a35f-27a608bad4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:56.755839 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b8b7e-3582-4be2-a35f-27a608bad4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:56.756112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b8b7e-3582-4be2-a35f-27a608bad4f9 None admin] PATH_INFO: `/v3/users/51325609aace494998b99ac3bce82c44` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:56.757400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b8b7e-3582-4be2-a35f-27a608bad4f9 None admin] RBAC: Authorizing `identity:get_user(user_id=51325609aace494998b99ac3bce82c44)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:56.759080 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b8b7e-3582-4be2-a35f-27a608bad4f9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:56.761643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 148/285] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:56 2022] GET /identity/v3/users/51325609aace494998b99ac3bce82c44 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:56.762555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b8b7e-3582-4be2-a35f-27a608bad4f9 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:56.817058 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:56.845006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456d24f-e2f7-4d76-a3b9-0ae31480f63c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:56.882890 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e05803-2e4b-40ac-8d16-78fa89aa81ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:56.883576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e05803-2e4b-40ac-8d16-78fa89aa81ae None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:56.883894 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e05803-2e4b-40ac-8d16-78fa89aa81ae None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:56.884200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e05803-2e4b-40ac-8d16-78fa89aa81ae None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:56.884949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e05803-2e4b-40ac-8d16-78fa89aa81ae None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:56.885307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e05803-2e4b-40ac-8d16-78fa89aa81ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:56.886723 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e05803-2e4b-40ac-8d16-78fa89aa81ae None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:56.895669 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b8b7e-3582-4be2-a35f-27a608bad4f9 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:56.899036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 138/286] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:19:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:56.900693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e05803-2e4b-40ac-8d16-78fa89aa81ae None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:56.938609 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b245e6-90c6-4064-87bb-9392bcfbd1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:56.940090 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b245e6-90c6-4064-87bb-9392bcfbd1d9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:56.940372 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b245e6-90c6-4064-87bb-9392bcfbd1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:56.940646 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b245e6-90c6-4064-87bb-9392bcfbd1d9 None admin] PATH_INFO: `/v3/users/55f7b374d05a42d6861c8913d521e327` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:56.957608 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b245e6-90c6-4064-87bb-9392bcfbd1d9 None admin] RBAC: Authorizing `identity:get_user(user_id=55f7b374d05a42d6861c8913d521e327)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:56.959679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b245e6-90c6-4064-87bb-9392bcfbd1d9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:56.962315 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 149/287] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:56 2022] GET /identity/v3/users/55f7b374d05a42d6861c8913d521e327 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:56.963170 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b245e6-90c6-4064-87bb-9392bcfbd1d9 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:57.024211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e05803-2e4b-40ac-8d16-78fa89aa81ae None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:57.037828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:57.080401 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925f2f3-0389-4d3c-97ea-11276bd7ca02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:57.081300 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925f2f3-0389-4d3c-97ea-11276bd7ca02 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:57.081729 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925f2f3-0389-4d3c-97ea-11276bd7ca02 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:57.082221 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925f2f3-0389-4d3c-97ea-11276bd7ca02 None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:57.083830 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925f2f3-0389-4d3c-97ea-11276bd7ca02 None admin] RBAC: Authorizing `identity:get_project(project_id=d659ef9d8a7e447cb15feaa6be1b2787)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:57.086766 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925f2f3-0389-4d3c-97ea-11276bd7ca02 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:57.090454 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 139/288] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:56 2022] GET /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:57.091881 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925f2f3-0389-4d3c-97ea-11276bd7ca02 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:57.105971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b245e6-90c6-4064-87bb-9392bcfbd1d9 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:57.109439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:57.153572 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1916ddb-9e61-4fea-9461-0bca54aa039a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:57.154652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1916ddb-9e61-4fea-9461-0bca54aa039a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:57.155165 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1916ddb-9e61-4fea-9461-0bca54aa039a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:57.155801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1916ddb-9e61-4fea-9461-0bca54aa039a None admin] PATH_INFO: `/v3/users/9d56df6133fd4e0d83b6090aa2abdcc5` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:57.157307 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1916ddb-9e61-4fea-9461-0bca54aa039a None admin] RBAC: Authorizing `identity:get_user(user_id=9d56df6133fd4e0d83b6090aa2abdcc5)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:57.159057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1916ddb-9e61-4fea-9461-0bca54aa039a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:57.160954 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 150/289] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:56 2022] GET /identity/v3/users/9d56df6133fd4e0d83b6090aa2abdcc5 => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:57.162078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1916ddb-9e61-4fea-9461-0bca54aa039a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:57.232523 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925f2f3-0389-4d3c-97ea-11276bd7ca02 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:57.273948 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c078c6-7404-47ce-8078-a64291b916d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:57.275208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c078c6-7404-47ce-8078-a64291b916d2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:57.276307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c078c6-7404-47ce-8078-a64291b916d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:57.276949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c078c6-7404-47ce-8078-a64291b916d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:57.278094 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c078c6-7404-47ce-8078-a64291b916d2 None admin] RBAC: Adding query filter params (role.id=9cda4e2827dc4c2e87477bef8f319f92, user.id=55f7b374d05a42d6861c8913d521e327, scope.system=all) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:57.278615 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c078c6-7404-47ce-8078-a64291b916d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:57.280554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c078c6-7404-47ce-8078-a64291b916d2 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:57.286481 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 140/290] 10.222.0.83 () {56 vars in 1308 bytes} [Mon Aug 1 15:19:57 2022] GET /identity/v3/role_assignments?user.id=55f7b374d05a42d6861c8913d521e327&scope.system=all&role.id=9cda4e2827dc4c2e87477bef8f319f92 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:57.287593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c078c6-7404-47ce-8078-a64291b916d2 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:57.295997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1916ddb-9e61-4fea-9461-0bca54aa039a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:57.358289 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:57.379577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9e1b6-5870-44f7-8c83-9b4bee310633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:57.380274 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9e1b6-5870-44f7-8c83-9b4bee310633 None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:57.380550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9e1b6-5870-44f7-8c83-9b4bee310633 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:57.380821 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9e1b6-5870-44f7-8c83-9b4bee310633 None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787/users/51325609aace494998b99ac3bce82c44/roles/fdbe7b923fed4101826cfb93a92e3fbf` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:57.384778 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9e1b6-5870-44f7-8c83-9b4bee310633 None admin] RBAC: Authorizing `identity:create_grant(project_id=d659ef9d8a7e447cb15feaa6be1b2787, user_id=51325609aace494998b99ac3bce82c44, role_id=fdbe7b923fed4101826cfb93a92e3fbf)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:57.386807 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9e1b6-5870-44f7-8c83-9b4bee310633 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:57.431336 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 151/291] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:19:57 2022] PUT /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787/users/51325609aace494998b99ac3bce82c44/roles/fdbe7b923fed4101826cfb93a92e3fbf => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:57.475310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:57.503213 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c078c6-7404-47ce-8078-a64291b916d2 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:57.546613 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78901b9-38f3-4951-a4be-f6aa3da88c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:57.547262 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78901b9-38f3-4951-a4be-f6aa3da88c53 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:57.547617 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78901b9-38f3-4951-a4be-f6aa3da88c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:57.547901 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78901b9-38f3-4951-a4be-f6aa3da88c53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:57.633800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78901b9-38f3-4951-a4be-f6aa3da88c53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:57.636024 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78901b9-38f3-4951-a4be-f6aa3da88c53 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:57.734344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:57.754932 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 141/292] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:19:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:19:57.761147 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9e1b6-5870-44f7-8c83-9b4bee310633 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:57.914239 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9e1b6-5870-44f7-8c83-9b4bee310633 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:57.957257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242f380-cba5-4b3a-a91c-1b998d6fb692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:57.957944 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242f380-cba5-4b3a-a91c-1b998d6fb692 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:57.958208 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242f380-cba5-4b3a-a91c-1b998d6fb692 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:57.958483 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242f380-cba5-4b3a-a91c-1b998d6fb692 None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:57.962586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242f380-cba5-4b3a-a91c-1b998d6fb692 None admin] RBAC: Authorizing `identity:get_project(project_id=d659ef9d8a7e447cb15feaa6be1b2787)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:57.964056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242f380-cba5-4b3a-a91c-1b998d6fb692 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:57.966060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 152/293] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:19:57 2022] GET /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:57.971844 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78901b9-38f3-4951-a4be-f6aa3da88c53 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:58.131332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78901b9-38f3-4951-a4be-f6aa3da88c53 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:58.173215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f92b15-6229-4957-89a1-0c3d16cb442e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:58.173837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f92b15-6229-4957-89a1-0c3d16cb442e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:58.174153 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f92b15-6229-4957-89a1-0c3d16cb442e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:58.174431 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f92b15-6229-4957-89a1-0c3d16cb442e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:58.175222 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f92b15-6229-4957-89a1-0c3d16cb442e None admin] RBAC: Adding query filter params (role.id=9cda4e2827dc4c2e87477bef8f319f92, scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787, user.id=9d56df6133fd4e0d83b6090aa2abdcc5) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:58.175536 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f92b15-6229-4957-89a1-0c3d16cb442e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:58.176996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f92b15-6229-4957-89a1-0c3d16cb442e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:58.183945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 142/294] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:19:57 2022] GET /identity/v3/role_assignments?user.id=9d56df6133fd4e0d83b6090aa2abdcc5&scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787&role.id=9cda4e2827dc4c2e87477bef8f319f92 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:58.740589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d90a5-74f6-44c3-9a97-b0fcbe7e5eb4 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:58.740881 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d90a5-74f6-44c3-9a97-b0fcbe7e5eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:58.741436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d90a5-74f6-44c3-9a97-b0fcbe7e5eb4 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:58.743338 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 153/295] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:58.748394 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0e529-3a10-4e4c-8394-9e649a7b8107 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:58.748950 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0e529-3a10-4e4c-8394-9e649a7b8107 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:58.749446 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0e529-3a10-4e4c-8394-9e649a7b8107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:58.768427 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d0d0e529-3a10-4e4c-8394-9e649a7b8107 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:58.770018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d0d0e529-3a10-4e4c-8394-9e649a7b8107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:58.802048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:58.845846 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 143/296] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:58.868841 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d90a5-74f6-44c3-9a97-b0fcbe7e5eb4 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:58.932546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:58.934459 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00e085-5af5-4b4a-bffa-178d4eeca496 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:58.935060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00e085-5af5-4b4a-bffa-178d4eeca496 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:58.935534 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00e085-5af5-4b4a-bffa-178d4eeca496 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:58.937370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 144/297] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:58.943081 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf0dc2-5e83-4327-ba0d-a90afb043b41 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:58.943650 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf0dc2-5e83-4327-ba0d-a90afb043b41 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:58.944124 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf0dc2-5e83-4327-ba0d-a90afb043b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:58.963151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-9ccf0dc2-5e83-4327-ba0d-a90afb043b41 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:58.964826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9ccf0dc2-5e83-4327-ba0d-a90afb043b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:58.986161 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d90a5-74f6-44c3-9a97-b0fcbe7e5eb4 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:59.028840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdfdba-416d-4aa8-a0af-5eb8d7e1809f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:59.029413 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdfdba-416d-4aa8-a0af-5eb8d7e1809f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:59.029689 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdfdba-416d-4aa8-a0af-5eb8d7e1809f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:59.029987 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdfdba-416d-4aa8-a0af-5eb8d7e1809f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:59.041994 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 145/298] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:19:59.042484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cdfdba-416d-4aa8-a0af-5eb8d7e1809f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:19:59.047440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-65cdfdba-416d-4aa8-a0af-5eb8d7e1809f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:19:59.047440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:19:59.047440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:19:59.047440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:19:59.047440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:19:59.047440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:19:59.047440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:19:59.047440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:19:59.047440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:19:59.047440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:19:59.047440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:19:59.047440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:19:59.047440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:19:59.047440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:19:59.047440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:19:59.047440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:19:59.048332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:19:59.048332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:19:59.048332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:19:59.048332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:19:59.048332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:19:59.048332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:19:59.048332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:19:59.048332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:19:59.048332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:19:59.048332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:19:59.048332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:19:59.048332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:19:59.048332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:19:59.048332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:19:59.048332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:19:59.048332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:19:59.049126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:19:59.049126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:19:59.049126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:19:59.049126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:19:59.049126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:19:59.049126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:19:59.049126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:19:59.049126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:19:59.049126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:19:59.049126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:19:59.049126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:19:59.049126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:19:59.049126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 154/299] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:19:59.057686 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf0dc2-5e83-4327-ba0d-a90afb043b41 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:59.077328 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdfdba-416d-4aa8-a0af-5eb8d7e1809f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:59.170995 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:59.180346 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf0dc2-5e83-4327-ba0d-a90afb043b41 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:59.206494 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdfdba-416d-4aa8-a0af-5eb8d7e1809f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:59.227509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91732e88-c854-4679-8983-2b1293860a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:59.227770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:59.228565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91732e88-c854-4679-8983-2b1293860a47 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:59.229732 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91732e88-c854-4679-8983-2b1293860a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:59.230396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91732e88-c854-4679-8983-2b1293860a47 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:59.231399 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91732e88-c854-4679-8983-2b1293860a47 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:59.231810 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91732e88-c854-4679-8983-2b1293860a47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:59.233348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91732e88-c854-4679-8983-2b1293860a47 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:59.245925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 146/300] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:19:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:59.249309 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91732e88-c854-4679-8983-2b1293860a47 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:59.260466 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077a646-3346-441d-8c8c-0162dfce18f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:59.261916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077a646-3346-441d-8c8c-0162dfce18f7 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:59.262390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077a646-3346-441d-8c8c-0162dfce18f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:59.263221 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077a646-3346-441d-8c8c-0162dfce18f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:59.264764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1077a646-3346-441d-8c8c-0162dfce18f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:59.267120 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1077a646-3346-441d-8c8c-0162dfce18f7 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:59.269755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 155/301] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:19:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:59.275666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077a646-3346-441d-8c8c-0162dfce18f7 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:59.382870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91732e88-c854-4679-8983-2b1293860a47 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:59.389977 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077a646-3346-441d-8c8c-0162dfce18f7 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:59.419443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2512b7f-60ce-468c-b695-5c2621bb4f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:59.420470 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2512b7f-60ce-468c-b695-5c2621bb4f8f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:59.420950 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2512b7f-60ce-468c-b695-5c2621bb4f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:59.421478 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2512b7f-60ce-468c-b695-5c2621bb4f8f None admin] PATH_INFO: `/v3/users/5b5245bf65024012a67523e633b8ef7b` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:59.433323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a2aa3-668b-48a7-a833-1ee794b40aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:59.433986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a2aa3-668b-48a7-a833-1ee794b40aa3 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:59.434272 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a2aa3-668b-48a7-a833-1ee794b40aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:59.434546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a2aa3-668b-48a7-a833-1ee794b40aa3 None admin] PATH_INFO: `/v3/groups` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:59.435381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a2aa3-668b-48a7-a833-1ee794b40aa3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:59.437357 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a2aa3-668b-48a7-a833-1ee794b40aa3 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:59.438756 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2512b7f-60ce-468c-b695-5c2621bb4f8f None admin] RBAC: Authorizing `identity:get_user(user_id=5b5245bf65024012a67523e633b8ef7b)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:59.440497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2512b7f-60ce-468c-b695-5c2621bb4f8f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:59.442869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 147/302] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:19:59 2022] GET /identity/v3/users/5b5245bf65024012a67523e633b8ef7b => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:59.449382 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2512b7f-60ce-468c-b695-5c2621bb4f8f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:59.466296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 156/303] 10.222.0.83 () {60 vars in 1134 bytes} [Mon Aug 1 15:19:59 2022] POST /identity/v3/groups => generated 229 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:59.474283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:59.574523 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2512b7f-60ce-468c-b695-5c2621bb4f8f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:59.617776 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28595313-3318-4a1c-90a6-847b12851003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:59.618452 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28595313-3318-4a1c-90a6-847b12851003 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:59.618739 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28595313-3318-4a1c-90a6-847b12851003 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:59.619016 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28595313-3318-4a1c-90a6-847b12851003 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:59.619879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28595313-3318-4a1c-90a6-847b12851003 None admin] RBAC: Adding query filter params (role.id=fdbe7b923fed4101826cfb93a92e3fbf, user.id=5b5245bf65024012a67523e633b8ef7b, scope.system=all) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:59.620224 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28595313-3318-4a1c-90a6-847b12851003 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:59.621767 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28595313-3318-4a1c-90a6-847b12851003 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:59.628217 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 148/304] 10.222.0.83 () {56 vars in 1308 bytes} [Mon Aug 1 15:19:59 2022] GET /identity/v3/role_assignments?user.id=5b5245bf65024012a67523e633b8ef7b&scope.system=all&role.id=fdbe7b923fed4101826cfb93a92e3fbf => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:59.791655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03122787-22ab-4800-a479-de7ff8806b45 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:59.792080 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03122787-22ab-4800-a479-de7ff8806b45 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:59.792479 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03122787-22ab-4800-a479-de7ff8806b45 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:59.793985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 157/305] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:59.799487 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8f80c-00af-4028-add6-901c7763607b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:59.800038 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8f80c-00af-4028-add6-901c7763607b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:59.800538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8f80c-00af-4028-add6-901c7763607b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:59.812934 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560c4d7-2569-4765-9aa3-50478444a196 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:59.813257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560c4d7-2569-4765-9aa3-50478444a196 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:59.813488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560c4d7-2569-4765-9aa3-50478444a196 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:59.815213 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 158/306] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:19:59.820722 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068b60b-afdf-4ed0-be38-1607979af0ae None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:59.821012 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068b60b-afdf-4ed0-be38-1607979af0ae None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:59.821292 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068b60b-afdf-4ed0-be38-1607979af0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:59.821639 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7be8f80c-00af-4028-add6-901c7763607b None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:59.823652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7be8f80c-00af-4028-add6-901c7763607b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:59.840618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1068b60b-afdf-4ed0-be38-1607979af0ae None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:59.841846 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1068b60b-afdf-4ed0-be38-1607979af0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:59.882807 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:59.909914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 149/307] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:19:59.934309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 159/308] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:19:59.938280 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8f80c-00af-4028-add6-901c7763607b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:59.954521 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068b60b-afdf-4ed0-be38-1607979af0ae None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:59.964082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:00.072036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8f80c-00af-4028-add6-901c7763607b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:00.095510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068b60b-afdf-4ed0-be38-1607979af0ae None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:00.120172 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf22b86-df0e-4218-9a8d-da38430a77a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:00.121777 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf22b86-df0e-4218-9a8d-da38430a77a1 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:00.122435 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf22b86-df0e-4218-9a8d-da38430a77a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:00.123004 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf22b86-df0e-4218-9a8d-da38430a77a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:00.135851 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6263619-1fa5-4b4d-834a-14362b8bac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:00.136495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6263619-1fa5-4b4d-834a-14362b8bac57 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:00.136772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6263619-1fa5-4b4d-834a-14362b8bac57 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:00.137043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6263619-1fa5-4b4d-834a-14362b8bac57 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:00.138483 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf22b86-df0e-4218-9a8d-da38430a77a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:00.145239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-7cf22b86-df0e-4218-9a8d-da38430a77a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:20:00.145239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:00.145239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:00.145239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:00.145239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:00.145239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:00.145239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:00.145239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:00.145239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:00.145239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:00.145239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:00.145239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:00.145239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:00.145239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:00.145239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:00.145239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:00.146163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:00.146163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:00.146163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:00.146163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:00.146163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:00.146163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:00.146163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:00.146163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:00.146163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:00.146163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:00.146163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:00.146163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:00.146163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:00.146163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:00.146163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:00.146163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:00.147053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:00.147053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:00.147053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:00.147053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:00.147053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:00.147053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:00.147053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:00.147053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:00.147053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:00.147053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:00.147053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:20:00.147053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:00.149332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6263619-1fa5-4b4d-834a-14362b8bac57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:00.150310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 150/309] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:00.161122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-c6263619-1fa5-4b4d-834a-14362b8bac57 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:20:00.161122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:00.161122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:00.161122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:00.161122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:00.161122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:00.161122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:00.161122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:00.161122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:00.161122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:00.161122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:00.161122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:00.161122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:00.161122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:00.161122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:00.161122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:00.162112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:00.162112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:00.162112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:00.162112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:00.162112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:00.162112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:00.162112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:00.162112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:00.162112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:00.162112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:00.162112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:00.162112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:00.162112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:00.162112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:00.162112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:00.162112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:00.162900 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:00.162900 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:00.162900 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:00.162900 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:00.162900 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:00.162900 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:00.162900 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:00.162900 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:00.162900 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:00.162900 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:00.162900 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:20:00.162900 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:00.163493 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf22b86-df0e-4218-9a8d-da38430a77a1 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:00.163568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 160/310] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:19:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:20:00.165135 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6263619-1fa5-4b4d-834a-14362b8bac57 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:00.205962 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:00.250179 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:00.288204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6263619-1fa5-4b4d-834a-14362b8bac57 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:00.294229 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf22b86-df0e-4218-9a8d-da38430a77a1 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:00.333001 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f66808-a691-4115-9584-41274077a631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:00.333636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f66808-a691-4115-9584-41274077a631 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:00.334295 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f66808-a691-4115-9584-41274077a631 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:00.334737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f66808-a691-4115-9584-41274077a631 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:00.335537 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f66808-a691-4115-9584-41274077a631 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:00.336171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f66808-a691-4115-9584-41274077a631 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:00.338333 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f66808-a691-4115-9584-41274077a631 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:00.342291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7455da7-1af7-41e7-8ab9-f7fdf3eaf7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:00.343205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7455da7-1af7-41e7-8ab9-f7fdf3eaf7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:00.343725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7455da7-1af7-41e7-8ab9-f7fdf3eaf7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:00.344190 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7455da7-1af7-41e7-8ab9-f7fdf3eaf7e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:00.345113 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7455da7-1af7-41e7-8ab9-f7fdf3eaf7e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:00.345596 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7455da7-1af7-41e7-8ab9-f7fdf3eaf7e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:00.347275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7455da7-1af7-41e7-8ab9-f7fdf3eaf7e5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:00.348781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 161/311] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:20:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:00.354284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f66808-a691-4115-9584-41274077a631 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:00.362013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 151/312] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:20:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:00.364790 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7455da7-1af7-41e7-8ab9-f7fdf3eaf7e5 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:00.458847 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:00.467443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f66808-a691-4115-9584-41274077a631 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:00.473440 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7455da7-1af7-41e7-8ab9-f7fdf3eaf7e5 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:00.503560 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8adbd-d6f0-41a9-9b34-12cfca9a56e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:00.504318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8adbd-d6f0-41a9-9b34-12cfca9a56e9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:00.504497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8adbd-d6f0-41a9-9b34-12cfca9a56e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:00.504766 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8adbd-d6f0-41a9-9b34-12cfca9a56e9 None admin] PATH_INFO: `/v3/users/51325609aace494998b99ac3bce82c44` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:00.505971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8adbd-d6f0-41a9-9b34-12cfca9a56e9 None admin] RBAC: Authorizing `identity:get_user(user_id=51325609aace494998b99ac3bce82c44)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:00.507426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8adbd-d6f0-41a9-9b34-12cfca9a56e9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:00.513835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 162/313] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:20:00 2022] GET /identity/v3/users/51325609aace494998b99ac3bce82c44 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:00.515352 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8adbd-d6f0-41a9-9b34-12cfca9a56e9 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:00.517281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bef45-24d5-49d1-93c6-50abf4d44a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:00.518270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bef45-24d5-49d1-93c6-50abf4d44a6b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:00.518755 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bef45-24d5-49d1-93c6-50abf4d44a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:00.519259 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bef45-24d5-49d1-93c6-50abf4d44a6b None admin] PATH_INFO: `/v3/users/55f7b374d05a42d6861c8913d521e327` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:00.520670 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bef45-24d5-49d1-93c6-50abf4d44a6b None admin] RBAC: Authorizing `identity:get_user(user_id=55f7b374d05a42d6861c8913d521e327)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:00.522416 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bef45-24d5-49d1-93c6-50abf4d44a6b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:00.524245 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 152/314] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:20:00 2022] GET /identity/v3/users/55f7b374d05a42d6861c8913d521e327 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:00.530117 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bef45-24d5-49d1-93c6-50abf4d44a6b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:00.533080 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:00.622978 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8adbd-d6f0-41a9-9b34-12cfca9a56e9 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:00.640773 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bef45-24d5-49d1-93c6-50abf4d44a6b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:00.664117 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083492ee-a3d1-401e-9ee9-d9e40c9928df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:00.665411 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083492ee-a3d1-401e-9ee9-d9e40c9928df None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:00.666741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083492ee-a3d1-401e-9ee9-d9e40c9928df None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:00.667736 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083492ee-a3d1-401e-9ee9-d9e40c9928df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:00.686512 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447f4bb-2537-4230-97f7-23551d13b405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:00.687273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447f4bb-2537-4230-97f7-23551d13b405 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:00.687560 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447f4bb-2537-4230-97f7-23551d13b405 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:00.687897 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447f4bb-2537-4230-97f7-23551d13b405 None admin] PATH_INFO: `/v3/system/users/55f7b374d05a42d6861c8913d521e327/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:00.689848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447f4bb-2537-4230-97f7-23551d13b405 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=55f7b374d05a42d6861c8913d521e327, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:00.692262 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447f4bb-2537-4230-97f7-23551d13b405 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:00.710773 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 153/315] 10.222.0.83 () {58 vars in 1334 bytes} [Mon Aug 1 15:20:00 2022] PUT /identity/v3/system/users/55f7b374d05a42d6861c8913d521e327/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:20:00.716544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:00.765929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083492ee-a3d1-401e-9ee9-d9e40c9928df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:00.767574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083492ee-a3d1-401e-9ee9-d9e40c9928df None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:00.887304 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 163/316] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:00.893483 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447f4bb-2537-4230-97f7-23551d13b405 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:00.967154 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:01.007199 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447f4bb-2537-4230-97f7-23551d13b405 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:01.043385 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db4b09-811c-4e96-8e0e-de82488c3f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:01.044043 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db4b09-811c-4e96-8e0e-de82488c3f7b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:01.044329 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db4b09-811c-4e96-8e0e-de82488c3f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:01.044607 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db4b09-811c-4e96-8e0e-de82488c3f7b None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:01.045833 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db4b09-811c-4e96-8e0e-de82488c3f7b None admin] RBAC: Authorizing `identity:get_project(project_id=d659ef9d8a7e447cb15feaa6be1b2787)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:01.047333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db4b09-811c-4e96-8e0e-de82488c3f7b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:01.049859 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 154/317] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:20:00 2022] GET /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:01.056066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083492ee-a3d1-401e-9ee9-d9e40c9928df None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:01.123604 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:01.165315 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083492ee-a3d1-401e-9ee9-d9e40c9928df None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:01.206274 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f721b-6eda-489d-b76d-a6271fcca2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:01.206906 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f721b-6eda-489d-b76d-a6271fcca2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:01.207278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f721b-6eda-489d-b76d-a6271fcca2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:01.207560 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f721b-6eda-489d-b76d-a6271fcca2e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:01.208416 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43f721b-6eda-489d-b76d-a6271fcca2e1 None admin] RBAC: Adding query filter params (role.id=fdbe7b923fed4101826cfb93a92e3fbf, scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787, user.id=51325609aace494998b99ac3bce82c44) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:01.208733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43f721b-6eda-489d-b76d-a6271fcca2e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:01.210264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43f721b-6eda-489d-b76d-a6271fcca2e1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:01.217194 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 164/318] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:20:01 2022] GET /identity/v3/role_assignments?user.id=51325609aace494998b99ac3bce82c44&scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787&role.id=fdbe7b923fed4101826cfb93a92e3fbf => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:01.638509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3128318-d865-4255-93bb-89c1169355fc None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:01.639045 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3128318-d865-4255-93bb-89c1169355fc None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:01.639045 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3128318-d865-4255-93bb-89c1169355fc None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:01.640541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 155/319] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:01.645553 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c7df0-f8ad-43d6-bff2-3dc31327157f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:01.645843 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c7df0-f8ad-43d6-bff2-3dc31327157f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:01.646144 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c7df0-f8ad-43d6-bff2-3dc31327157f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:01.664343 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e92c7df0-f8ad-43d6-bff2-3dc31327157f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:01.665524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e92c7df0-f8ad-43d6-bff2-3dc31327157f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:01.710661 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a99bf-8143-4b67-b307-ccbe09de10d4 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:01.710946 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a99bf-8143-4b67-b307-ccbe09de10d4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:01.711242 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a99bf-8143-4b67-b307-ccbe09de10d4 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:01.712701 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 156/320] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:20:01.717423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e28852-94bd-431d-a9ab-fd46086dfdbe None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:01.717719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e28852-94bd-431d-a9ab-fd46086dfdbe None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:01.718034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e28852-94bd-431d-a9ab-fd46086dfdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:01.736043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 165/321] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:01.736351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c8e28852-94bd-431d-a9ab-fd46086dfdbe None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:01.737546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c8e28852-94bd-431d-a9ab-fd46086dfdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:01.758299 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c7df0-f8ad-43d6-bff2-3dc31327157f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:01.808988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 157/322] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:20:01.818295 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:01.831925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e28852-94bd-431d-a9ab-fd46086dfdbe None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:01.867519 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c7df0-f8ad-43d6-bff2-3dc31327157f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:01.901934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:01.908455 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4073b79-4ad0-4cf9-a22a-685ae89ca84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:01.909165 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4073b79-4ad0-4cf9-a22a-685ae89ca84e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:01.909450 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4073b79-4ad0-4cf9-a22a-685ae89ca84e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:01.909724 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4073b79-4ad0-4cf9-a22a-685ae89ca84e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:01.910895 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4073b79-4ad0-4cf9-a22a-685ae89ca84e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:01.912358 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4073b79-4ad0-4cf9-a22a-685ae89ca84e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:01.919011 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 166/323] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:01.919994 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4073b79-4ad0-4cf9-a22a-685ae89ca84e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:01.945860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e28852-94bd-431d-a9ab-fd46086dfdbe None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:01.983506 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5f065-a081-493b-a41c-c50e49c1eed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:01.984164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5f065-a081-493b-a41c-c50e49c1eed1 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:01.984450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5f065-a081-493b-a41c-c50e49c1eed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:01.984728 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5f065-a081-493b-a41c-c50e49c1eed1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:01.994840 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5f065-a081-493b-a41c-c50e49c1eed1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:02.000131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-eee5f065-a081-493b-a41c-c50e49c1eed1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:20:02.000131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:02.000131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:02.000131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:02.000131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:02.000131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:02.000131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:02.000131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:02.000131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:02.000131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:02.000131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:02.000131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:02.000131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:02.000131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:02.000131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:02.000131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:02.001047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:02.001047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:02.001047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:02.001047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:02.001047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:02.001047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:02.001047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:02.001047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:02.001047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:02.001047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:02.001047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:02.001047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:02.001047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:02.001047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:02.001047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:02.001047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:02.001988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:02.001988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:02.001988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:02.001988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:02.001988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:02.001988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:02.001988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:02.001988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:02.001988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:02.001988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:02.001988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:20:02.001988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:02.001988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 158/324] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:20:02.007002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5f065-a081-493b-a41c-c50e49c1eed1 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:02.036648 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4073b79-4ad0-4cf9-a22a-685ae89ca84e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:02.064702 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:02.077854 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a77b35-8b98-457a-a8ec-8d10282d17f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:02.078631 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a77b35-8b98-457a-a8ec-8d10282d17f9 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:02.078919 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a77b35-8b98-457a-a8ec-8d10282d17f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:02.079197 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a77b35-8b98-457a-a8ec-8d10282d17f9 None admin] PATH_INFO: `/v3/groups` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:02.079863 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a77b35-8b98-457a-a8ec-8d10282d17f9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:02.081304 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a77b35-8b98-457a-a8ec-8d10282d17f9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:02.100689 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 167/325] 10.222.0.83 () {60 vars in 1134 bytes} [Mon Aug 1 15:20:01 2022] POST /identity/v3/groups => generated 226 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:20:02.118982 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5f065-a081-493b-a41c-c50e49c1eed1 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:02.146580 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:02.160778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83113ef-21b3-4422-b748-4791b06e202e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:02.161341 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83113ef-21b3-4422-b748-4791b06e202e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:02.161627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83113ef-21b3-4422-b748-4791b06e202e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:02.161922 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83113ef-21b3-4422-b748-4791b06e202e None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:02.162680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83113ef-21b3-4422-b748-4791b06e202e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:02.162989 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83113ef-21b3-4422-b748-4791b06e202e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:02.164805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83113ef-21b3-4422-b748-4791b06e202e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:02.174594 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 159/326] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:20:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:02.180350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a77b35-8b98-457a-a8ec-8d10282d17f9 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:02.292845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a77b35-8b98-457a-a8ec-8d10282d17f9 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:02.329491 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf223f-3d5d-4ab0-a27c-6cf313335c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:02.330172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf223f-3d5d-4ab0-a27c-6cf313335c5d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:02.330459 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf223f-3d5d-4ab0-a27c-6cf313335c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:02.330735 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf223f-3d5d-4ab0-a27c-6cf313335c5d None admin] PATH_INFO: `/v3/users/5b5245bf65024012a67523e633b8ef7b` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:02.331921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf223f-3d5d-4ab0-a27c-6cf313335c5d None admin] RBAC: Authorizing `identity:get_user(user_id=5b5245bf65024012a67523e633b8ef7b)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:02.333377 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf223f-3d5d-4ab0-a27c-6cf313335c5d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:02.335214 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 168/327] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:20:02 2022] GET /identity/v3/users/5b5245bf65024012a67523e633b8ef7b => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:02.340707 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83113ef-21b3-4422-b748-4791b06e202e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:02.454521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83113ef-21b3-4422-b748-4791b06e202e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:02.491959 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe926777-acbb-48f7-9e6c-3cf9bdf9d075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:02.492637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe926777-acbb-48f7-9e6c-3cf9bdf9d075 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:02.492927 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe926777-acbb-48f7-9e6c-3cf9bdf9d075 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:02.493208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe926777-acbb-48f7-9e6c-3cf9bdf9d075 None admin] PATH_INFO: `/v3/system/users/5b5245bf65024012a67523e633b8ef7b/roles/fdbe7b923fed4101826cfb93a92e3fbf` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:02.495259 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe926777-acbb-48f7-9e6c-3cf9bdf9d075 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5b5245bf65024012a67523e633b8ef7b, role_id=fdbe7b923fed4101826cfb93a92e3fbf)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:02.496675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe926777-acbb-48f7-9e6c-3cf9bdf9d075 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:02.505447 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 160/328] 10.222.0.83 () {58 vars in 1334 bytes} [Mon Aug 1 15:20:02 2022] PUT /identity/v3/system/users/5b5245bf65024012a67523e633b8ef7b/roles/fdbe7b923fed4101826cfb93a92e3fbf => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:20:02.782715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbc74a-adb7-43a9-9691-4f7b0a920766 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:02.783054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbc74a-adb7-43a9-9691-4f7b0a920766 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:02.783231 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbc74a-adb7-43a9-9691-4f7b0a920766 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:02.784843 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 169/329] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:02.790272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0863c9-2a5e-4736-a16e-fed80411a338 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:02.790713 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0863c9-2a5e-4736-a16e-fed80411a338 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:02.790814 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0863c9-2a5e-4736-a16e-fed80411a338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:02.810426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-cb0863c9-2a5e-4736-a16e-fed80411a338 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:02.811623 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-cb0863c9-2a5e-4736-a16e-fed80411a338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:02.864630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:02.892859 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 161/330] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:02.911368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbc74a-adb7-43a9-9691-4f7b0a920766 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:02.976550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:03.038226 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbc74a-adb7-43a9-9691-4f7b0a920766 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:03.087052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b0212-c8f6-4574-9446-955e15e2774a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:03.087708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b0212-c8f6-4574-9446-955e15e2774a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:03.088559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b0212-c8f6-4574-9446-955e15e2774a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:03.088838 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b0212-c8f6-4574-9446-955e15e2774a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:03.102506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2b0212-c8f6-4574-9446-955e15e2774a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:03.109172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-7b2b0212-c8f6-4574-9446-955e15e2774a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:20:03.109172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:03.109172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:03.109172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:03.109172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:03.109172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:03.109172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:03.109172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:03.109172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:03.109172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:03.109172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:03.109172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:03.109172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:03.109172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:03.109172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:03.109172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:03.110107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:03.110107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:03.110107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:03.110107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:03.110107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:03.110107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:03.110107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:03.110107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:03.110107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:03.110107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:03.110107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:03.110107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:03.110107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:03.110107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:03.110107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:03.110107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:03.110894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:03.110894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:03.110894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:03.110894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:03.110894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:03.110894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:03.110894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:03.110894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:03.110894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:03.110894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:03.110894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:20:03.110894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:03.110894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 170/331] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:03.116467 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0863c9-2a5e-4736-a16e-fed80411a338 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:03.271041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0863c9-2a5e-4736-a16e-fed80411a338 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:03.314866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c358ec3-e5ca-4fe0-8631-01f82b5d7274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:03.315422 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c358ec3-e5ca-4fe0-8631-01f82b5d7274 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:03.315706 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c358ec3-e5ca-4fe0-8631-01f82b5d7274 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:03.315982 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c358ec3-e5ca-4fe0-8631-01f82b5d7274 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:03.316706 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c358ec3-e5ca-4fe0-8631-01f82b5d7274 None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:03.317012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c358ec3-e5ca-4fe0-8631-01f82b5d7274 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:03.318427 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c358ec3-e5ca-4fe0-8631-01f82b5d7274 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:03.327505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 162/332] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:20:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:03.334974 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b0212-c8f6-4574-9446-955e15e2774a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:03.480155 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b0212-c8f6-4574-9446-955e15e2774a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:03.498771 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:03.524966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339956ef-7799-4975-91d8-461d7d6440c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:03.526326 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339956ef-7799-4975-91d8-461d7d6440c6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:03.526649 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339956ef-7799-4975-91d8-461d7d6440c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:03.526927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339956ef-7799-4975-91d8-461d7d6440c6 None admin] PATH_INFO: `/v3/users/55f7b374d05a42d6861c8913d521e327` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:03.528144 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339956ef-7799-4975-91d8-461d7d6440c6 None admin] RBAC: Authorizing `identity:get_user(user_id=55f7b374d05a42d6861c8913d521e327)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:03.531897 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339956ef-7799-4975-91d8-461d7d6440c6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:03.533745 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 171/333] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:20:03 2022] GET /identity/v3/users/55f7b374d05a42d6861c8913d521e327 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:03.539723 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c358ec3-e5ca-4fe0-8631-01f82b5d7274 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:03.609405 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:03.674519 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c358ec3-e5ca-4fe0-8631-01f82b5d7274 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:03.712064 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7f536-ca1d-4be1-a915-34240eaa98b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:03.713139 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7f536-ca1d-4be1-a915-34240eaa98b5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:03.713626 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7f536-ca1d-4be1-a915-34240eaa98b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:03.714147 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7f536-ca1d-4be1-a915-34240eaa98b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:03.715146 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7f536-ca1d-4be1-a915-34240eaa98b5 None admin] RBAC: Adding query filter params (role.id=9cda4e2827dc4c2e87477bef8f319f92, user.id=55f7b374d05a42d6861c8913d521e327, scope.system=all) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:03.715660 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7f536-ca1d-4be1-a915-34240eaa98b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:03.717341 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7f536-ca1d-4be1-a915-34240eaa98b5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:03.733998 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 163/334] 10.222.0.83 () {56 vars in 1308 bytes} [Mon Aug 1 15:20:03 2022] GET /identity/v3/role_assignments?user.id=55f7b374d05a42d6861c8913d521e327&scope.system=all&role.id=9cda4e2827dc4c2e87477bef8f319f92 => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:04.753494 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4305d4-4025-4618-bd91-4666d64f41e5 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:04.754042 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4305d4-4025-4618-bd91-4666d64f41e5 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:04.754116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4305d4-4025-4618-bd91-4666d64f41e5 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:04.755767 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 172/335] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:04.762042 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae31e2-e0b9-4fb6-855c-952baef8a2bd None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:04.762495 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae31e2-e0b9-4fb6-855c-952baef8a2bd None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:04.762778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae31e2-e0b9-4fb6-855c-952baef8a2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:04.786954 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-45ae31e2-e0b9-4fb6-855c-952baef8a2bd None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:04.788170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-45ae31e2-e0b9-4fb6-855c-952baef8a2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:04.877105 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 164/336] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:20:04.923416 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4305d4-4025-4618-bd91-4666d64f41e5 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:05.054014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4305d4-4025-4618-bd91-4666d64f41e5 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:05.099438 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08aff9-2149-4415-be5f-69d2f40e7e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:05.100102 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08aff9-2149-4415-be5f-69d2f40e7e1c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:05.100388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08aff9-2149-4415-be5f-69d2f40e7e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:05.100664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08aff9-2149-4415-be5f-69d2f40e7e1c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:05.112461 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc08aff9-2149-4415-be5f-69d2f40e7e1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:05.117513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-bc08aff9-2149-4415-be5f-69d2f40e7e1c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:20:05.117513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:05.117513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:05.117513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:05.117513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:05.117513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:05.117513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:05.117513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:05.117513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:05.117513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:05.117513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:05.117513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:05.117513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:05.117513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:05.117513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:05.117513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:05.118439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:05.118439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:05.118439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:05.118439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:05.118439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:05.118439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:05.118439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:05.118439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:05.118439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:05.118439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:05.118439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:05.118439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:05.118439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:05.118439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:05.118439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:05.118439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:05.119801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:05.119801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:05.119801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:05.119801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:05.119801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:05.119801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:05.119801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:05.119801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:05.119801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:05.119801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:05.119801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:20:05.119801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:05.119801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 173/337] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:05.125525 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae31e2-e0b9-4fb6-855c-952baef8a2bd None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:05.170762 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:05.257733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae31e2-e0b9-4fb6-855c-952baef8a2bd None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:05.301460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece7cd5-1235-4355-90c0-4cb0301e4137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:05.302122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece7cd5-1235-4355-90c0-4cb0301e4137 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:05.302411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece7cd5-1235-4355-90c0-4cb0301e4137 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:05.302687 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece7cd5-1235-4355-90c0-4cb0301e4137 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:05.303472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece7cd5-1235-4355-90c0-4cb0301e4137 None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:05.303726 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece7cd5-1235-4355-90c0-4cb0301e4137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:05.305136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece7cd5-1235-4355-90c0-4cb0301e4137 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:05.318005 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 165/338] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:20:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:05.321468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08aff9-2149-4415-be5f-69d2f40e7e1c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:05.384655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:05.455835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08aff9-2149-4415-be5f-69d2f40e7e1c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:05.479565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237229e-a07f-495b-bf66-1c77c84cac11 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:05.480002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237229e-a07f-495b-bf66-1c77c84cac11 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:05.482409 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237229e-a07f-495b-bf66-1c77c84cac11 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:05.485229 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 166/339] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:05.489834 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c897f-f2e5-45ad-b3cd-abf1f72f5599 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:05.490159 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c897f-f2e5-45ad-b3cd-abf1f72f5599 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:05.490444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c897f-f2e5-45ad-b3cd-abf1f72f5599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:05.494719 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c3087-1199-4598-aa3d-bb7f9a97051c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:05.495386 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c3087-1199-4598-aa3d-bb7f9a97051c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:05.495690 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c3087-1199-4598-aa3d-bb7f9a97051c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:05.495949 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c3087-1199-4598-aa3d-bb7f9a97051c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:05.520815 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3d9c897f-f2e5-45ad-b3cd-abf1f72f5599 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:05.526278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3d9c897f-f2e5-45ad-b3cd-abf1f72f5599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:05.574593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c3087-1199-4598-aa3d-bb7f9a97051c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:05.576203 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c3087-1199-4598-aa3d-bb7f9a97051c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:05.621446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:05.637108 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 167/340] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:05.685811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 174/341] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:05.686994 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c897f-f2e5-45ad-b3cd-abf1f72f5599 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:05.698783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c3087-1199-4598-aa3d-bb7f9a97051c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:05.719341 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:05.833946 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c897f-f2e5-45ad-b3cd-abf1f72f5599 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:05.846758 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c3087-1199-4598-aa3d-bb7f9a97051c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:05.885194 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d557e-6141-43a5-9eaf-af6e521eb5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:05.885855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d557e-6141-43a5-9eaf-af6e521eb5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:05.886263 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d557e-6141-43a5-9eaf-af6e521eb5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:05.886541 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d557e-6141-43a5-9eaf-af6e521eb5b9 None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:05.886783 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a866047-02b1-497f-8ce7-9ab46b80ea7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:05.887312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a866047-02b1-497f-8ce7-9ab46b80ea7b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:05.887598 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a866047-02b1-497f-8ce7-9ab46b80ea7b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:05.887775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d557e-6141-43a5-9eaf-af6e521eb5b9 None admin] RBAC: Authorizing `identity:get_project(project_id=44e0498d623545e68ea89aa6de0a8be7)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:05.887934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a866047-02b1-497f-8ce7-9ab46b80ea7b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:05.889443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d557e-6141-43a5-9eaf-af6e521eb5b9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:05.897640 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 175/342] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:20:05 2022] GET /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:05.899321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb24fc-77a0-47dd-a03a-d0374dfaa8ca None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:05.899624 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb24fc-77a0-47dd-a03a-d0374dfaa8ca None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:05.899905 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb24fc-77a0-47dd-a03a-d0374dfaa8ca None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:05.901853 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 176/343] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:05.902690 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb24fc-77a0-47dd-a03a-d0374dfaa8ca None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:05.905105 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a866047-02b1-497f-8ce7-9ab46b80ea7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:05.910719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-5a866047-02b1-497f-8ce7-9ab46b80ea7b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:20:05.910719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:05.910719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:05.910719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:05.910719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:05.910719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:05.910719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:05.910719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:05.910719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:05.910719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:05.910719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:05.910719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:05.910719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:05.910719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:05.910719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:05.910719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:05.912888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:05.912888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:05.912888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:05.912888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:05.912888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:05.912888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:05.912888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:05.912888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:05.912888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:05.912888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:05.912888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:05.912888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:05.912888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:05.912888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:05.912888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:05.912888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:05.914939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:05.914939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:05.914939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:05.914939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:05.914939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:05.914939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:05.914939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:05.914939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:05.914939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:05.914939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:05.914939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:20:05.914939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:05.914939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 168/344] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:20:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:05.914939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a752a1b-b737-4bae-ad1b-b172334424f6 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:05.917257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:05.917400 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a752a1b-b737-4bae-ad1b-b172334424f6 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:05.917400 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a752a1b-b737-4bae-ad1b-b172334424f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:05.940579 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-0a752a1b-b737-4bae-ad1b-b172334424f6 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:05.943167 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0a752a1b-b737-4bae-ad1b-b172334424f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:06.020406 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 169/345] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:06.021578 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a752a1b-b737-4bae-ad1b-b172334424f6 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:06.028532 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb24fc-77a0-47dd-a03a-d0374dfaa8ca None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:06.037605 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:06.069875 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b78f5-f778-4112-9f12-5b5378f5ae03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:06.072181 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b78f5-f778-4112-9f12-5b5378f5ae03 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:06.072934 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b78f5-f778-4112-9f12-5b5378f5ae03 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:06.073225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b78f5-f778-4112-9f12-5b5378f5ae03 None admin] PATH_INFO: `/v3/groups/42a3c71258c541149e49dbe26f21fad1` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:06.079874 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b78f5-f778-4112-9f12-5b5378f5ae03 None admin] RBAC: Authorizing `identity:get_group(group_id=42a3c71258c541149e49dbe26f21fad1)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:06.081354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b78f5-f778-4112-9f12-5b5378f5ae03 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:06.083423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 177/346] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Aug 1 15:20:05 2022] GET /identity/v3/groups/42a3c71258c541149e49dbe26f21fad1 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:06.084665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f47f8-b9a0-4a85-a8a1-9763a9816207 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:06.084895 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f47f8-b9a0-4a85-a8a1-9763a9816207 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:06.085176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f47f8-b9a0-4a85-a8a1-9763a9816207 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:06.090516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 178/347] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:06.091282 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f47f8-b9a0-4a85-a8a1-9763a9816207 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:06.143145 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a752a1b-b737-4bae-ad1b-b172334424f6 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:06.174384 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:06.185659 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b8b1c-340a-4901-b452-d3363d6eed8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:06.186386 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b8b1c-340a-4901-b452-d3363d6eed8a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:06.186693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b8b1c-340a-4901-b452-d3363d6eed8a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:06.187004 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b8b1c-340a-4901-b452-d3363d6eed8a None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:06.187849 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73b8b1c-340a-4901-b452-d3363d6eed8a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:06.188295 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73b8b1c-340a-4901-b452-d3363d6eed8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:06.189760 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73b8b1c-340a-4901-b452-d3363d6eed8a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:06.199602 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 170/348] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:06.200642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b8b1c-340a-4901-b452-d3363d6eed8a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:06.208165 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f47f8-b9a0-4a85-a8a1-9763a9816207 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:06.244559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f4f9d-0d11-4b42-a301-83ae7b909afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:06.245214 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f4f9d-0d11-4b42-a301-83ae7b909afb None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:06.245499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f4f9d-0d11-4b42-a301-83ae7b909afb None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:06.245835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f4f9d-0d11-4b42-a301-83ae7b909afb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:06.257235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f4f9d-0d11-4b42-a301-83ae7b909afb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:06.262579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-1a7f4f9d-0d11-4b42-a301-83ae7b909afb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:20:06.262579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:06.262579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:06.262579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:06.262579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:06.262579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:06.262579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:06.262579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:06.262579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:06.262579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:06.262579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:06.262579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:06.262579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:06.262579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:06.262579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:06.262579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:06.264679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:06.264679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:06.264679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:06.264679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:06.264679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:06.264679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:06.264679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:06.264679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:06.264679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:06.264679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:06.264679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:06.264679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:06.264679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:06.264679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:06.264679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:06.264679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:06.266647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:06.266647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:06.266647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:06.266647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:06.266647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:06.266647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:06.266647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:06.266647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:06.266647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:06.266647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:06.266647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:20:06.266647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:06.266647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 179/349] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:06.266647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8831cd-1b14-4cc7-9dde-16831771892e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:06.268730 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8831cd-1b14-4cc7-9dde-16831771892e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:06.268730 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8831cd-1b14-4cc7-9dde-16831771892e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:06.285489 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-be8831cd-1b14-4cc7-9dde-16831771892e None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:06.286788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-be8831cd-1b14-4cc7-9dde-16831771892e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:06.287235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:06.317484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b8b1c-340a-4901-b452-d3363d6eed8a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:06.360328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230aa776-d6b9-48ed-bd9a-a9022c164f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:06.361190 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230aa776-d6b9-48ed-bd9a-a9022c164f6a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:06.361640 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230aa776-d6b9-48ed-bd9a-a9022c164f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:06.362153 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230aa776-d6b9-48ed-bd9a-a9022c164f6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:06.363179 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 180/350] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:06.363518 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230aa776-d6b9-48ed-bd9a-a9022c164f6a None admin] RBAC: Adding query filter params (group.id=42a3c71258c541149e49dbe26f21fad1, role.id=9cda4e2827dc4c2e87477bef8f319f92, scope.project.id=44e0498d623545e68ea89aa6de0a8be7) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:06.363977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230aa776-d6b9-48ed-bd9a-a9022c164f6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:06.364105 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8831cd-1b14-4cc7-9dde-16831771892e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:06.366280 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230aa776-d6b9-48ed-bd9a-a9022c164f6a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:06.373910 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 171/351] 10.222.0.83 () {56 vars in 1376 bytes} [Mon Aug 1 15:20:06 2022] GET /identity/v3/role_assignments?group.id=42a3c71258c541149e49dbe26f21fad1&scope.project.id=44e0498d623545e68ea89aa6de0a8be7&role.id=9cda4e2827dc4c2e87477bef8f319f92 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:06.374946 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230aa776-d6b9-48ed-bd9a-a9022c164f6a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:06.478845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:06.521593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8831cd-1b14-4cc7-9dde-16831771892e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:06.564475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529ed3a-daf6-4331-8dd0-ce3f15c759e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:06.565116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529ed3a-daf6-4331-8dd0-ce3f15c759e0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:06.565402 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529ed3a-daf6-4331-8dd0-ce3f15c759e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:06.565680 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529ed3a-daf6-4331-8dd0-ce3f15c759e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:06.572062 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230aa776-d6b9-48ed-bd9a-a9022c164f6a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:06.641179 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4147d1-a57b-4a2d-8a59-1877151de9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:06.642207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4147d1-a57b-4a2d-8a59-1877151de9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:06.642698 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4147d1-a57b-4a2d-8a59-1877151de9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:06.643176 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4147d1-a57b-4a2d-8a59-1877151de9a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:06.644137 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529ed3a-daf6-4331-8dd0-ce3f15c759e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:06.644251 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4147d1-a57b-4a2d-8a59-1877151de9a6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:06.644745 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4147d1-a57b-4a2d-8a59-1877151de9a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:06.645714 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529ed3a-daf6-4331-8dd0-ce3f15c759e0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:06.646668 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4147d1-a57b-4a2d-8a59-1877151de9a6 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:06.658127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:06.667929 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 172/352] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:20:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:06.670291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4147d1-a57b-4a2d-8a59-1877151de9a6 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:06.718463 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:06.761336 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 181/353] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:06.762079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529ed3a-daf6-4331-8dd0-ce3f15c759e0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:06.792236 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4147d1-a57b-4a2d-8a59-1877151de9a6 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:06.846459 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf507f6-8e75-4525-8c69-4442fb4d7b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:06.847926 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf507f6-8e75-4525-8c69-4442fb4d7b25 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:06.848415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf507f6-8e75-4525-8c69-4442fb4d7b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:06.848987 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf507f6-8e75-4525-8c69-4442fb4d7b25 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:06.865679 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf507f6-8e75-4525-8c69-4442fb4d7b25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:06.872060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-2bf507f6-8e75-4525-8c69-4442fb4d7b25 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:20:06.872060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:06.872060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:06.872060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:06.872060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:06.872060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:06.872060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:06.872060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:06.872060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:06.872060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:06.872060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:06.872060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:06.872060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:06.872060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:06.872060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:06.872060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:06.872964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:06.872964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:06.872964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:06.872964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:06.872964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:06.872964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:06.872964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:06.872964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:06.872964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:06.872964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:06.872964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:06.872964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:06.872964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:06.872964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:06.872964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:06.872964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:06.873827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:06.873827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:06.873827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:06.873827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:06.873827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:06.873827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:06.873827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:06.873827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:06.873827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:06.873827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:06.873827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:20:06.873827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:06.876532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 173/354] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:06.881260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529ed3a-daf6-4331-8dd0-ce3f15c759e0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:06.882863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf507f6-8e75-4525-8c69-4442fb4d7b25 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:06.917997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126dc1e-fb46-48fa-8e87-58c13107843f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:06.918457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126dc1e-fb46-48fa-8e87-58c13107843f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:06.918741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126dc1e-fb46-48fa-8e87-58c13107843f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:06.919015 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126dc1e-fb46-48fa-8e87-58c13107843f None admin] PATH_INFO: `/v3/users/5b5245bf65024012a67523e633b8ef7b` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:06.920227 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126dc1e-fb46-48fa-8e87-58c13107843f None admin] RBAC: Authorizing `identity:get_user(user_id=5b5245bf65024012a67523e633b8ef7b)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:06.921715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126dc1e-fb46-48fa-8e87-58c13107843f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:06.925734 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 182/355] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:20:06 2022] GET /identity/v3/users/5b5245bf65024012a67523e633b8ef7b => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:06.925734 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126dc1e-fb46-48fa-8e87-58c13107843f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:06.958157 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:06.975926 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:07.033376 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126dc1e-fb46-48fa-8e87-58c13107843f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:07.034339 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf507f6-8e75-4525-8c69-4442fb4d7b25 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:07.088977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e5902-273d-4b97-b69c-b0399331ffee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:07.089343 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8da20-41dd-4ee9-b9e1-f3ec8d787ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:07.089925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e5902-273d-4b97-b69c-b0399331ffee None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:07.090328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e5902-273d-4b97-b69c-b0399331ffee None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:07.090524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8da20-41dd-4ee9-b9e1-f3ec8d787ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:07.090800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e5902-273d-4b97-b69c-b0399331ffee None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:07.091124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8da20-41dd-4ee9-b9e1-f3ec8d787ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:07.091558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8da20-41dd-4ee9-b9e1-f3ec8d787ce6 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:07.092317 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e5902-273d-4b97-b69c-b0399331ffee None admin] RBAC: Authorizing `identity:get_project(project_id=44e0498d623545e68ea89aa6de0a8be7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:07.092609 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b8da20-41dd-4ee9-b9e1-f3ec8d787ce6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:07.093065 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b8da20-41dd-4ee9-b9e1-f3ec8d787ce6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:07.094120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e5902-273d-4b97-b69c-b0399331ffee None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:07.094864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b8da20-41dd-4ee9-b9e1-f3ec8d787ce6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:07.096029 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 174/356] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:20:06 2022] GET /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:07.096960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e5902-273d-4b97-b69c-b0399331ffee None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:07.106495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 183/357] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:20:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:07.108711 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4083621-5b84-44e0-b783-33c043812a83 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:07.109170 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4083621-5b84-44e0-b783-33c043812a83 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:07.110126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4083621-5b84-44e0-b783-33c043812a83 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:07.112434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 184/358] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:07.113371 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4083621-5b84-44e0-b783-33c043812a83 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:07.216353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e5902-273d-4b97-b69c-b0399331ffee None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:07.232683 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:07.248410 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4083621-5b84-44e0-b783-33c043812a83 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:07.255244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:07.275610 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5a8eb-eed8-4329-b5e2-62189709287d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:07.276859 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5a8eb-eed8-4329-b5e2-62189709287d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:07.277385 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5a8eb-eed8-4329-b5e2-62189709287d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:07.278083 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5a8eb-eed8-4329-b5e2-62189709287d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:07.279115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5a8eb-eed8-4329-b5e2-62189709287d None admin] RBAC: Adding query filter params (role.id=fdbe7b923fed4101826cfb93a92e3fbf, user.id=5b5245bf65024012a67523e633b8ef7b, scope.system=all) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:07.279477 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5a8eb-eed8-4329-b5e2-62189709287d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:07.281413 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5a8eb-eed8-4329-b5e2-62189709287d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:07.302828 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 175/359] 10.222.0.83 () {56 vars in 1308 bytes} [Mon Aug 1 15:20:07 2022] GET /identity/v3/role_assignments?user.id=5b5245bf65024012a67523e633b8ef7b&scope.system=all&role.id=fdbe7b923fed4101826cfb93a92e3fbf => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:07.302828 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5a8eb-eed8-4329-b5e2-62189709287d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:07.323639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cba4d9-6616-4787-a286-1ddd6812fff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:07.324012 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cba4d9-6616-4787-a286-1ddd6812fff8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:07.324433 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cba4d9-6616-4787-a286-1ddd6812fff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:07.324895 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cba4d9-6616-4787-a286-1ddd6812fff8 None admin] PATH_INFO: `/v3/groups/42a3c71258c541149e49dbe26f21fad1` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:07.327791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cba4d9-6616-4787-a286-1ddd6812fff8 None admin] RBAC: Authorizing `identity:get_group(group_id=42a3c71258c541149e49dbe26f21fad1)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:07.332042 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cba4d9-6616-4787-a286-1ddd6812fff8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:07.336492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 185/360] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Aug 1 15:20:07 2022] GET /identity/v3/groups/42a3c71258c541149e49dbe26f21fad1 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:07.339744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b4d43-6539-4e09-90b2-9d7946377120 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:07.340232 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b4d43-6539-4e09-90b2-9d7946377120 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:07.340735 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b4d43-6539-4e09-90b2-9d7946377120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:07.376826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e57b4d43-6539-4e09-90b2-9d7946377120 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:07.378734 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e57b4d43-6539-4e09-90b2-9d7946377120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:07.501583 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5a8eb-eed8-4329-b5e2-62189709287d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:07.510580 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 186/361] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:07.511561 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba31a06-e177-4f8d-aa96-0596ce660278 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:07.511857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba31a06-e177-4f8d-aa96-0596ce660278 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:07.512241 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba31a06-e177-4f8d-aa96-0596ce660278 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:07.515763 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 187/362] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:07.517244 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba31a06-e177-4f8d-aa96-0596ce660278 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:07.560094 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b86cc8-a503-4173-a070-e57bea8136c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:07.561303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b86cc8-a503-4173-a070-e57bea8136c5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:07.561862 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b86cc8-a503-4173-a070-e57bea8136c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:07.562448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b86cc8-a503-4173-a070-e57bea8136c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:07.635608 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:07.648281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:07.659080 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b86cc8-a503-4173-a070-e57bea8136c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:07.660411 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba31a06-e177-4f8d-aa96-0596ce660278 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:07.660703 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b86cc8-a503-4173-a070-e57bea8136c5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:07.703137 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c973c-baf1-4951-9d7d-da71ab3e46b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:07.703764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c973c-baf1-4951-9d7d-da71ab3e46b1 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:07.704048 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c973c-baf1-4951-9d7d-da71ab3e46b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:07.704326 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c973c-baf1-4951-9d7d-da71ab3e46b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:07.705146 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c973c-baf1-4951-9d7d-da71ab3e46b1 None admin] RBAC: Adding query filter params (group.id=42a3c71258c541149e49dbe26f21fad1, role.id=73e29e4f51e6469facf94aaa698c5cfe, scope.project.id=44e0498d623545e68ea89aa6de0a8be7) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:07.705471 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c973c-baf1-4951-9d7d-da71ab3e46b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:07.707312 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c973c-baf1-4951-9d7d-da71ab3e46b1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:07.716312 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 188/363] 10.222.0.83 () {56 vars in 1376 bytes} [Mon Aug 1 15:20:07 2022] GET /identity/v3/role_assignments?group.id=42a3c71258c541149e49dbe26f21fad1&scope.project.id=44e0498d623545e68ea89aa6de0a8be7&role.id=73e29e4f51e6469facf94aaa698c5cfe => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:07.717915 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7fec5-0c25-4442-800e-cc8ea4ae532e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:07.718269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7fec5-0c25-4442-800e-cc8ea4ae532e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:07.718551 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7fec5-0c25-4442-800e-cc8ea4ae532e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:07.739485 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f7b7fec5-0c25-4442-800e-cc8ea4ae532e None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:07.740749 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f7b7fec5-0c25-4442-800e-cc8ea4ae532e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:07.802751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 176/364] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:07.803569 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b86cc8-a503-4173-a070-e57bea8136c5 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:07.849637 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 189/365] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:07.850745 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7fec5-0c25-4442-800e-cc8ea4ae532e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:07.995978 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:08.028951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b86cc8-a503-4173-a070-e57bea8136c5 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:08.035500 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:08.060708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7fec5-0c25-4442-800e-cc8ea4ae532e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:08.073843 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebc57f-6e61-475e-8452-d581ca06972a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:08.074609 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebc57f-6e61-475e-8452-d581ca06972a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:08.074895 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebc57f-6e61-475e-8452-d581ca06972a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:08.075178 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebc57f-6e61-475e-8452-d581ca06972a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:08.086470 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cebc57f-6e61-475e-8452-d581ca06972a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:08.093260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-0cebc57f-6e61-475e-8452-d581ca06972a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:20:08.093260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:08.093260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:08.093260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:08.093260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:08.093260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:08.093260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:08.093260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:08.093260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:08.093260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:08.093260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:08.093260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:08.093260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:08.093260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:08.093260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:08.093260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:08.094272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:08.094272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:08.094272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:08.094272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:08.094272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:08.094272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:08.094272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:08.094272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:08.094272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:08.094272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:08.094272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:08.094272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:08.094272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:08.094272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:08.094272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:08.094272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:08.095574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:08.095574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:08.095574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:08.095574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:08.095574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:08.095574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:08.095574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:08.095574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:08.095574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:08.095574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:08.095574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:20:08.095574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:08.095574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 177/366] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:20:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:08.097902 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebc57f-6e61-475e-8452-d581ca06972a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:08.108205 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adeac4e-9585-44dd-894c-23baa8bcf73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:08.108938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adeac4e-9585-44dd-894c-23baa8bcf73d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:08.109226 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adeac4e-9585-44dd-894c-23baa8bcf73d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:08.109535 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adeac4e-9585-44dd-894c-23baa8bcf73d None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:08.110900 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adeac4e-9585-44dd-894c-23baa8bcf73d None admin] RBAC: Authorizing `identity:get_project(project_id=d659ef9d8a7e447cb15feaa6be1b2787)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:08.113000 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adeac4e-9585-44dd-894c-23baa8bcf73d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:08.115138 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 190/367] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:20:07 2022] GET /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787 => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:08.115941 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adeac4e-9585-44dd-894c-23baa8bcf73d None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:08.252824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adeac4e-9585-44dd-894c-23baa8bcf73d None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:08.253292 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebc57f-6e61-475e-8452-d581ca06972a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:08.302407 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70be3e-ba2f-4d64-aaf6-93d609a0ab35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:08.302800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4732c-46b7-4efa-930d-2f54dcb09ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:08.303049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70be3e-ba2f-4d64-aaf6-93d609a0ab35 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:08.303334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70be3e-ba2f-4d64-aaf6-93d609a0ab35 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:08.303519 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4732c-46b7-4efa-930d-2f54dcb09ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:08.303679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70be3e-ba2f-4d64-aaf6-93d609a0ab35 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:08.303828 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4732c-46b7-4efa-930d-2f54dcb09ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:08.304577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4732c-46b7-4efa-930d-2f54dcb09ab6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:08.304828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70be3e-ba2f-4d64-aaf6-93d609a0ab35 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:08.304828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70be3e-ba2f-4d64-aaf6-93d609a0ab35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:08.306431 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70be3e-ba2f-4d64-aaf6-93d609a0ab35 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:08.317215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4732c-46b7-4efa-930d-2f54dcb09ab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:08.318388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 191/368] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:08.319313 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70be3e-ba2f-4d64-aaf6-93d609a0ab35 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:08.323398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-fae4732c-46b7-4efa-930d-2f54dcb09ab6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:20:08.323398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:08.323398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:08.323398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:08.323398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:08.323398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:08.323398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:08.323398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:08.323398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:08.323398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:08.323398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:08.323398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:08.323398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:08.323398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:08.323398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:08.323398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:08.325541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:08.325541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:08.325541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:08.325541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:08.325541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:08.325541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:08.325541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:08.325541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:08.325541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:08.325541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:08.325541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:08.325541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:08.325541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:08.325541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:08.325541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:08.325541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:08.327642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:08.327642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:08.327642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:08.327642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:08.327642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:08.327642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:08.327642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:08.327642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:08.327642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:08.327642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:08.327642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:20:08.327642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:08.327642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 178/369] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:20:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:08.337920 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4732c-46b7-4efa-930d-2f54dcb09ab6 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:08.338800 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:08.341386 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:08.469734 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4732c-46b7-4efa-930d-2f54dcb09ab6 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:08.472991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70be3e-ba2f-4d64-aaf6-93d609a0ab35 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:08.520202 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cd021-b1a9-499c-8573-5a8f5b2409d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:08.520895 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cd021-b1a9-499c-8573-5a8f5b2409d1 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:08.521192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cd021-b1a9-499c-8573-5a8f5b2409d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:08.521466 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cd021-b1a9-499c-8573-5a8f5b2409d1 None admin] PATH_INFO: `/v3/groups/42a3c71258c541149e49dbe26f21fad1` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:08.522677 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95cd021-b1a9-499c-8573-5a8f5b2409d1 None admin] RBAC: Authorizing `identity:get_group(group_id=42a3c71258c541149e49dbe26f21fad1)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:08.524140 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95cd021-b1a9-499c-8573-5a8f5b2409d1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:08.525998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 192/370] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Aug 1 15:20:08 2022] GET /identity/v3/groups/42a3c71258c541149e49dbe26f21fad1 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:08.526781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cd021-b1a9-499c-8573-5a8f5b2409d1 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:08.559920 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582c262-fac6-4c43-9e58-418725c4835a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:08.560537 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582c262-fac6-4c43-9e58-418725c4835a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:08.560929 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582c262-fac6-4c43-9e58-418725c4835a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:08.561215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582c262-fac6-4c43-9e58-418725c4835a None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:08.561987 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582c262-fac6-4c43-9e58-418725c4835a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:08.562302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582c262-fac6-4c43-9e58-418725c4835a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:08.563768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582c262-fac6-4c43-9e58-418725c4835a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:08.574907 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 179/371] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:08.575769 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582c262-fac6-4c43-9e58-418725c4835a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:08.674850 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:08.698514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cd021-b1a9-499c-8573-5a8f5b2409d1 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:08.700347 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:08.749235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582c262-fac6-4c43-9e58-418725c4835a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:08.776426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206efa68-585d-4531-82e9-a42897dd46c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:08.777778 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206efa68-585d-4531-82e9-a42897dd46c1 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:08.778301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206efa68-585d-4531-82e9-a42897dd46c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:08.779061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206efa68-585d-4531-82e9-a42897dd46c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:08.793500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628656e-15d0-4bf4-b589-f5a17bd3f5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:08.794218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628656e-15d0-4bf4-b589-f5a17bd3f5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:08.794505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628656e-15d0-4bf4-b589-f5a17bd3f5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:08.794838 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628656e-15d0-4bf4-b589-f5a17bd3f5a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:08.795654 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1628656e-15d0-4bf4-b589-f5a17bd3f5a7 None admin] RBAC: Adding query filter params (group.id=42a3c71258c541149e49dbe26f21fad1, role.id=9cda4e2827dc4c2e87477bef8f319f92, scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:08.795967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1628656e-15d0-4bf4-b589-f5a17bd3f5a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:08.797565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1628656e-15d0-4bf4-b589-f5a17bd3f5a7 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:08.804945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 180/372] 10.222.0.83 () {56 vars in 1376 bytes} [Mon Aug 1 15:20:08 2022] GET /identity/v3/role_assignments?group.id=42a3c71258c541149e49dbe26f21fad1&scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787&role.id=9cda4e2827dc4c2e87477bef8f319f92 => generated 260 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:08.805798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628656e-15d0-4bf4-b589-f5a17bd3f5a7 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:08.943962 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206efa68-585d-4531-82e9-a42897dd46c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:08.948019 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206efa68-585d-4531-82e9-a42897dd46c1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:08.954664 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628656e-15d0-4bf4-b589-f5a17bd3f5a7 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:09.003379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:09.012482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bcd33-ddae-4070-928c-3ffa7a10eb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:09.013137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bcd33-ddae-4070-928c-3ffa7a10eb5c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:09.013425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bcd33-ddae-4070-928c-3ffa7a10eb5c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:09.013703 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bcd33-ddae-4070-928c-3ffa7a10eb5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:09.063183 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:09.087046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 193/373] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:09.090081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7f5be-81e6-4edb-8de3-221ec7dc30d3 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:09.090708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7f5be-81e6-4edb-8de3-221ec7dc30d3 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:09.091237 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7f5be-81e6-4edb-8de3-221ec7dc30d3 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:09.093693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 194/374] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:09.095544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7f5be-81e6-4edb-8de3-221ec7dc30d3 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:09.126494 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8bcd33-ddae-4070-928c-3ffa7a10eb5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:09.128993 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8bcd33-ddae-4070-928c-3ffa7a10eb5c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:09.260212 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 181/375] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:09.262482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf33ef1-c907-4fa1-a3e9-3a41dd5fe8dd None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:09.262482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf33ef1-c907-4fa1-a3e9-3a41dd5fe8dd None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:09.262482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf33ef1-c907-4fa1-a3e9-3a41dd5fe8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:09.268932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7f5be-81e6-4edb-8de3-221ec7dc30d3 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:09.291889 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3bf33ef1-c907-4fa1-a3e9-3a41dd5fe8dd None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:09.294255 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3bf33ef1-c907-4fa1-a3e9-3a41dd5fe8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:09.317569 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b78f98-07f1-4ffd-8dce-92aa900c8c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:09.318436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b78f98-07f1-4ffd-8dce-92aa900c8c53 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:09.318752 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b78f98-07f1-4ffd-8dce-92aa900c8c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:09.319100 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b78f98-07f1-4ffd-8dce-92aa900c8c53 None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:09.321047 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b78f98-07f1-4ffd-8dce-92aa900c8c53 None admin] RBAC: Authorizing `identity:get_project(project_id=d659ef9d8a7e447cb15feaa6be1b2787)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:09.322957 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b78f98-07f1-4ffd-8dce-92aa900c8c53 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:09.329350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 195/376] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:20:09 2022] GET /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:09.329350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b78f98-07f1-4ffd-8dce-92aa900c8c53 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:09.342157 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:09.395788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:09.410504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 182/377] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:09.412403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf33ef1-c907-4fa1-a3e9-3a41dd5fe8dd None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:09.458930 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b78f98-07f1-4ffd-8dce-92aa900c8c53 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:09.511292 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c46f5-94c3-4bd5-89d1-54b43687965a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:09.512018 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c46f5-94c3-4bd5-89d1-54b43687965a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:09.512414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c46f5-94c3-4bd5-89d1-54b43687965a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:09.512693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c46f5-94c3-4bd5-89d1-54b43687965a None admin] PATH_INFO: `/v3/projects/6c0cb293301d4dcdbc4412b1e11a284f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:09.518568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c46f5-94c3-4bd5-89d1-54b43687965a None admin] RBAC: Authorizing `identity:get_project(project_id=6c0cb293301d4dcdbc4412b1e11a284f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:09.518568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c46f5-94c3-4bd5-89d1-54b43687965a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:09.520049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 196/378] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:20:09 2022] GET /identity/v3/projects/6c0cb293301d4dcdbc4412b1e11a284f => generated 344 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:09.520623 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c46f5-94c3-4bd5-89d1-54b43687965a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:09.562313 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf33ef1-c907-4fa1-a3e9-3a41dd5fe8dd None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:09.625732 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8783ee-33db-4793-832c-8203bc8b451e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:09.627000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8783ee-33db-4793-832c-8203bc8b451e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:09.627551 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8783ee-33db-4793-832c-8203bc8b451e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:09.628176 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8783ee-33db-4793-832c-8203bc8b451e None admin] PATH_INFO: `/v3/groups/42a3c71258c541149e49dbe26f21fad1` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:09.629996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8783ee-33db-4793-832c-8203bc8b451e None admin] RBAC: Authorizing `identity:get_group(group_id=42a3c71258c541149e49dbe26f21fad1)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:09.634448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8783ee-33db-4793-832c-8203bc8b451e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:09.637363 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 183/379] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Aug 1 15:20:09 2022] GET /identity/v3/groups/42a3c71258c541149e49dbe26f21fad1 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:09.638587 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8783ee-33db-4793-832c-8203bc8b451e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:09.674647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c46f5-94c3-4bd5-89d1-54b43687965a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:09.690893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:09.698200 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:09.717587 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fffab82-ac91-45a3-858f-70fa3b4df9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:09.718518 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffab82-ac91-45a3-858f-70fa3b4df9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:09.718922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffab82-ac91-45a3-858f-70fa3b4df9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:09.719372 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffab82-ac91-45a3-858f-70fa3b4df9c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:09.731039 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fffab82-ac91-45a3-858f-70fa3b4df9c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:09.736305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-8fffab82-ac91-45a3-858f-70fa3b4df9c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:20:09.736305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:09.736305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:09.736305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:09.736305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:09.736305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:09.736305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:09.736305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:09.736305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:09.736305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:09.736305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:09.736305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:09.736305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:09.736305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:09.736305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:09.736305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:09.737403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:09.737403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:09.737403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:09.737403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:09.737403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:09.737403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:09.737403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:09.737403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:09.737403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:09.737403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:09.737403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:09.737403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:09.737403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:09.737403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:09.737403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:09.737403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:09.738229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:09.738229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:09.738229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:09.738229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:09.738229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:09.738229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:09.738229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:09.738229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:09.738229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:09.738229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:09.738229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:20:09.738229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:09.739033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 197/380] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:09.740057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fffab82-ac91-45a3-858f-70fa3b4df9c7 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:09.790279 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8783ee-33db-4793-832c-8203bc8b451e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:09.840457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410420bd-afd0-4664-901d-1881fdede21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:09.841474 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410420bd-afd0-4664-901d-1881fdede21c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:09.841934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410420bd-afd0-4664-901d-1881fdede21c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:09.842355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410420bd-afd0-4664-901d-1881fdede21c None admin] PATH_INFO: `/v3/groups/856ad0fafa8d472ca81e7380077787c5` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:09.849613 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410420bd-afd0-4664-901d-1881fdede21c None admin] RBAC: Authorizing `identity:get_group(group_id=856ad0fafa8d472ca81e7380077787c5)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:09.851576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410420bd-afd0-4664-901d-1881fdede21c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:09.853957 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 184/381] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Aug 1 15:20:09 2022] GET /identity/v3/groups/856ad0fafa8d472ca81e7380077787c5 => generated 229 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:09.855363 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410420bd-afd0-4664-901d-1881fdede21c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:09.857969 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fffab82-ac91-45a3-858f-70fa3b4df9c7 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:09.905459 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be2896-79f8-403e-ac0d-a9fa80216af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:09.906330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be2896-79f8-403e-ac0d-a9fa80216af0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:09.906754 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be2896-79f8-403e-ac0d-a9fa80216af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:09.907149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be2896-79f8-403e-ac0d-a9fa80216af0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:09.908414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be2896-79f8-403e-ac0d-a9fa80216af0 None admin] RBAC: Adding query filter params (group.id=42a3c71258c541149e49dbe26f21fad1, role.id=73e29e4f51e6469facf94aaa698c5cfe, scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:09.908836 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be2896-79f8-403e-ac0d-a9fa80216af0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:09.911490 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be2896-79f8-403e-ac0d-a9fa80216af0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:09.920106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 198/382] 10.222.0.83 () {56 vars in 1376 bytes} [Mon Aug 1 15:20:09 2022] GET /identity/v3/role_assignments?group.id=42a3c71258c541149e49dbe26f21fad1&scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787&role.id=73e29e4f51e6469facf94aaa698c5cfe => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:09.920106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be2896-79f8-403e-ac0d-a9fa80216af0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:09.995233 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:10.055933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:10.076294 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410420bd-afd0-4664-901d-1881fdede21c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:10.088521 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be2896-79f8-403e-ac0d-a9fa80216af0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:10.128036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48ac82-9b12-4794-a818-00d855da27db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:10.128675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48ac82-9b12-4794-a818-00d855da27db None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:10.128963 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48ac82-9b12-4794-a818-00d855da27db None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:10.129236 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48ac82-9b12-4794-a818-00d855da27db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:10.130079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48ac82-9b12-4794-a818-00d855da27db None admin] RBAC: Adding query filter params (group.id=856ad0fafa8d472ca81e7380077787c5, role.id=7bdfa43dc1d046f7965ffe52b74bc58f, scope.project.id=6c0cb293301d4dcdbc4412b1e11a284f) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:10.130395 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48ac82-9b12-4794-a818-00d855da27db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:10.131905 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48ac82-9b12-4794-a818-00d855da27db None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:10.140038 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 199/383] 10.222.0.83 () {56 vars in 1376 bytes} [Mon Aug 1 15:20:09 2022] GET /identity/v3/role_assignments?group.id=856ad0fafa8d472ca81e7380077787c5&scope.project.id=6c0cb293301d4dcdbc4412b1e11a284f&role.id=7bdfa43dc1d046f7965ffe52b74bc58f => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:10.182423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba835980-fa74-4c5e-82de-4a2291ade338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:10.183097 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba835980-fa74-4c5e-82de-4a2291ade338 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:10.183377 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba835980-fa74-4c5e-82de-4a2291ade338 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:10.183754 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba835980-fa74-4c5e-82de-4a2291ade338 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:10.184516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba835980-fa74-4c5e-82de-4a2291ade338 None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:10.184831 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba835980-fa74-4c5e-82de-4a2291ade338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:10.186295 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba835980-fa74-4c5e-82de-4a2291ade338 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:10.205182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 185/384] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:20:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:10.210964 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48ac82-9b12-4794-a818-00d855da27db None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:10.322703 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac363ff-61b5-4d1f-a55c-00c39bbda71f None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:10.322993 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac363ff-61b5-4d1f-a55c-00c39bbda71f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:10.323272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac363ff-61b5-4d1f-a55c-00c39bbda71f None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:10.325177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 186/385] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:10.334986 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871ce3e-b049-4776-a63f-1f9f0243e8ff None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:10.335346 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871ce3e-b049-4776-a63f-1f9f0243e8ff None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:10.335651 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871ce3e-b049-4776-a63f-1f9f0243e8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:10.356068 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48ac82-9b12-4794-a818-00d855da27db None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:10.364824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d871ce3e-b049-4776-a63f-1f9f0243e8ff None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:10.366539 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d871ce3e-b049-4776-a63f-1f9f0243e8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:10.369645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:10.421710 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb67aca-066d-4e1d-9b5a-87cf82eb14d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:10.422524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb67aca-066d-4e1d-9b5a-87cf82eb14d0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:10.422922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb67aca-066d-4e1d-9b5a-87cf82eb14d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:10.423211 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb67aca-066d-4e1d-9b5a-87cf82eb14d0 None admin] PATH_INFO: `/v3/groups/42a3c71258c541149e49dbe26f21fad1` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:10.425245 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb67aca-066d-4e1d-9b5a-87cf82eb14d0 None admin] RBAC: Authorizing `identity:get_group(group_id=42a3c71258c541149e49dbe26f21fad1)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:10.428799 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb67aca-066d-4e1d-9b5a-87cf82eb14d0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:10.435460 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 200/386] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Aug 1 15:20:10 2022] GET /identity/v3/groups/42a3c71258c541149e49dbe26f21fad1 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:10.442836 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb67aca-066d-4e1d-9b5a-87cf82eb14d0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:10.506482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 187/387] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:10.540627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871ce3e-b049-4776-a63f-1f9f0243e8ff None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:10.636564 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb67aca-066d-4e1d-9b5a-87cf82eb14d0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:10.686877 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:10.699244 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ddd69-4cf3-41cf-bc8e-0067384a0352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:10.699486 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871ce3e-b049-4776-a63f-1f9f0243e8ff None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:10.699913 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ddd69-4cf3-41cf-bc8e-0067384a0352 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:10.700185 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ddd69-4cf3-41cf-bc8e-0067384a0352 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:10.700454 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ddd69-4cf3-41cf-bc8e-0067384a0352 None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:10.704015 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ddd69-4cf3-41cf-bc8e-0067384a0352 None admin] RBAC: Authorizing `identity:get_project(project_id=44e0498d623545e68ea89aa6de0a8be7)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:10.705983 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ddd69-4cf3-41cf-bc8e-0067384a0352 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:10.708262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 201/388] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:20:10 2022] GET /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:10.716902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ddd69-4cf3-41cf-bc8e-0067384a0352 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:10.782361 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79f41a-2cb2-441c-96c7-8b27d4f2f9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:10.783252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79f41a-2cb2-441c-96c7-8b27d4f2f9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:10.783533 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79f41a-2cb2-441c-96c7-8b27d4f2f9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:10.783806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79f41a-2cb2-441c-96c7-8b27d4f2f9c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:10.806496 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79f41a-2cb2-441c-96c7-8b27d4f2f9c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:10.811554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-8c79f41a-2cb2-441c-96c7-8b27d4f2f9c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:20:10.811554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:10.811554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:10.811554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:10.811554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:10.811554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:10.811554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:10.811554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:10.811554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:10.811554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:10.811554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:10.811554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:10.811554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:10.811554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:10.811554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:10.811554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:10.812516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:10.812516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:10.812516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:10.812516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:10.812516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:10.812516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:10.812516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:10.812516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:10.812516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:10.812516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:10.812516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:10.812516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:10.812516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:10.812516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:10.812516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:10.812516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:10.813366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:10.813366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:10.813366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:10.813366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:10.813366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:10.813366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:10.813366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:10.813366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:10.813366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:10.813366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:10.813366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:20:10.813366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:10.813366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 188/389] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:20:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:20:10.814923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:10.821243 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79f41a-2cb2-441c-96c7-8b27d4f2f9c8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:10.942651 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ddd69-4cf3-41cf-bc8e-0067384a0352 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:11.014180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e6917-34a2-49ce-9639-ea36baedff68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:11.014873 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e6917-34a2-49ce-9639-ea36baedff68 None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:11.015165 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e6917-34a2-49ce-9639-ea36baedff68 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:11.015445 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e6917-34a2-49ce-9639-ea36baedff68 None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7/groups/42a3c71258c541149e49dbe26f21fad1/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:11.022719 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e6917-34a2-49ce-9639-ea36baedff68 None admin] RBAC: Authorizing `identity:create_grant(project_id=44e0498d623545e68ea89aa6de0a8be7, group_id=42a3c71258c541149e49dbe26f21fad1, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:11.024195 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e6917-34a2-49ce-9639-ea36baedff68 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:11.032667 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79f41a-2cb2-441c-96c7-8b27d4f2f9c8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:11.051665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 202/390] 10.222.0.83 () {58 vars in 1442 bytes} [Mon Aug 1 15:20:10 2022] PUT /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7/groups/42a3c71258c541149e49dbe26f21fad1/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:20:11.098666 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58d50c-aceb-4455-8920-592a69e0da12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:11.099378 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58d50c-aceb-4455-8920-592a69e0da12 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:11.099674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58d50c-aceb-4455-8920-592a69e0da12 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:11.099963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58d50c-aceb-4455-8920-592a69e0da12 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:11.100776 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58d50c-aceb-4455-8920-592a69e0da12 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:11.101089 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58d50c-aceb-4455-8920-592a69e0da12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:11.102622 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58d50c-aceb-4455-8920-592a69e0da12 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:11.115019 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 189/391] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:11.121330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e6917-34a2-49ce-9639-ea36baedff68 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:11.256388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e6917-34a2-49ce-9639-ea36baedff68 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:11.258073 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:11.302542 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480718a7-445d-452c-b5a5-d9e77adbdc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:11.303205 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480718a7-445d-452c-b5a5-d9e77adbdc3c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:11.303538 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480718a7-445d-452c-b5a5-d9e77adbdc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:11.303826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480718a7-445d-452c-b5a5-d9e77adbdc3c None admin] PATH_INFO: `/v3/groups/42a3c71258c541149e49dbe26f21fad1` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:11.305041 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480718a7-445d-452c-b5a5-d9e77adbdc3c None admin] RBAC: Authorizing `identity:get_group(group_id=42a3c71258c541149e49dbe26f21fad1)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:11.307061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480718a7-445d-452c-b5a5-d9e77adbdc3c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:11.308683 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 203/392] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Aug 1 15:20:11 2022] GET /identity/v3/groups/42a3c71258c541149e49dbe26f21fad1 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:11.314522 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58d50c-aceb-4455-8920-592a69e0da12 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:11.327762 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:11.443079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58d50c-aceb-4455-8920-592a69e0da12 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:11.491173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba4fc6-c64a-4aaf-9e8e-1958ac4d84c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:11.491906 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba4fc6-c64a-4aaf-9e8e-1958ac4d84c5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:11.492197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba4fc6-c64a-4aaf-9e8e-1958ac4d84c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:11.492519 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba4fc6-c64a-4aaf-9e8e-1958ac4d84c5 None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:11.498859 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba4fc6-c64a-4aaf-9e8e-1958ac4d84c5 None admin] RBAC: Authorizing `identity:get_project(project_id=44e0498d623545e68ea89aa6de0a8be7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:11.500353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba4fc6-c64a-4aaf-9e8e-1958ac4d84c5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:11.502648 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 190/393] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:20:11 2022] GET /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:11.508594 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480718a7-445d-452c-b5a5-d9e77adbdc3c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:11.631306 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480718a7-445d-452c-b5a5-d9e77adbdc3c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:11.675237 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42ea38-012b-4445-abe6-233805cb328d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:11.675942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42ea38-012b-4445-abe6-233805cb328d None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:11.676225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42ea38-012b-4445-abe6-233805cb328d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:11.676507 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42ea38-012b-4445-abe6-233805cb328d None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7/groups/42a3c71258c541149e49dbe26f21fad1/roles/73e29e4f51e6469facf94aaa698c5cfe` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:11.678818 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42ea38-012b-4445-abe6-233805cb328d None admin] RBAC: Authorizing `identity:create_grant(project_id=44e0498d623545e68ea89aa6de0a8be7, group_id=42a3c71258c541149e49dbe26f21fad1, role_id=73e29e4f51e6469facf94aaa698c5cfe)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:11.680284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42ea38-012b-4445-abe6-233805cb328d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:11.696889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 204/394] 10.222.0.83 () {58 vars in 1442 bytes} [Mon Aug 1 15:20:11 2022] PUT /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7/groups/42a3c71258c541149e49dbe26f21fad1/roles/73e29e4f51e6469facf94aaa698c5cfe => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:20:11.857194 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c21fef-75f4-49a0-8ed6-f47658163558 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:11.857710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c21fef-75f4-49a0-8ed6-f47658163558 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:11.858159 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c21fef-75f4-49a0-8ed6-f47658163558 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:11.860033 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 191/395] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:11.869748 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803060cc-a3ab-4320-97f8-5b68e8d090f6 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:11.870323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803060cc-a3ab-4320-97f8-5b68e8d090f6 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:11.870759 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803060cc-a3ab-4320-97f8-5b68e8d090f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:11.893205 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-803060cc-a3ab-4320-97f8-5b68e8d090f6 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:11.894824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-803060cc-a3ab-4320-97f8-5b68e8d090f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:11.929062 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:11.989412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 205/396] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:12.016247 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c21fef-75f4-49a0-8ed6-f47658163558 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:12.109311 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:12.140966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c21fef-75f4-49a0-8ed6-f47658163558 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:12.194396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512634f-e0b4-4523-9b9d-2cf6998f975a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:12.195370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512634f-e0b4-4523-9b9d-2cf6998f975a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:12.195655 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512634f-e0b4-4523-9b9d-2cf6998f975a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:12.195929 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512634f-e0b4-4523-9b9d-2cf6998f975a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:12.208241 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2512634f-e0b4-4523-9b9d-2cf6998f975a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:12.215936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-2512634f-e0b4-4523-9b9d-2cf6998f975a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:20:12.215936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:12.215936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:12.215936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:12.215936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:12.215936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:12.215936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:12.215936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:12.215936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:12.215936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:12.215936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:12.215936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:12.215936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:12.215936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:12.215936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:12.215936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:12.217128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:12.217128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:12.217128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:12.217128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:12.217128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:12.217128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:12.217128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:12.217128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:12.217128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:12.217128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:12.217128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:12.217128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:12.217128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:12.217128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:12.217128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:12.217128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:12.218055 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:12.218055 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:12.218055 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:12.218055 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:12.218055 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:12.218055 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:12.218055 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:12.218055 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:12.218055 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:12.218055 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:12.218055 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:20:12.218055 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:12.218055 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 192/397] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:12.223207 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803060cc-a3ab-4320-97f8-5b68e8d090f6 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:12.355452 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803060cc-a3ab-4320-97f8-5b68e8d090f6 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:12.411720 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a0cb2-46af-4f24-8518-7f7e50b58e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:12.412378 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a0cb2-46af-4f24-8518-7f7e50b58e7a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:12.412724 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a0cb2-46af-4f24-8518-7f7e50b58e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:12.413002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a0cb2-46af-4f24-8518-7f7e50b58e7a None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:12.413797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a0cb2-46af-4f24-8518-7f7e50b58e7a None admin] RBAC: Adding query filter params (name=member) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:12.414133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a0cb2-46af-4f24-8518-7f7e50b58e7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:12.415514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a0cb2-46af-4f24-8518-7f7e50b58e7a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:12.427409 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 206/398] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:20:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:12.433090 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512634f-e0b4-4523-9b9d-2cf6998f975a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:12.578594 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512634f-e0b4-4523-9b9d-2cf6998f975a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:12.651367 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efba5d-1728-4fae-91f3-589e2cae2087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:12.652480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efba5d-1728-4fae-91f3-589e2cae2087 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:12.652952 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efba5d-1728-4fae-91f3-589e2cae2087 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:12.653407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efba5d-1728-4fae-91f3-589e2cae2087 None admin] PATH_INFO: `/v3/groups/42a3c71258c541149e49dbe26f21fad1` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:12.659042 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81efba5d-1728-4fae-91f3-589e2cae2087 None admin] RBAC: Authorizing `identity:get_group(group_id=42a3c71258c541149e49dbe26f21fad1)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:12.661615 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81efba5d-1728-4fae-91f3-589e2cae2087 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:12.664661 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 193/399] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Aug 1 15:20:12 2022] GET /identity/v3/groups/42a3c71258c541149e49dbe26f21fad1 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:12.670842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a0cb2-46af-4f24-8518-7f7e50b58e7a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:12.718647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:12.866888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a0cb2-46af-4f24-8518-7f7e50b58e7a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:12.892084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb3c9d-e5c1-4388-a800-46c29f2f2cf5 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:12.892918 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb3c9d-e5c1-4388-a800-46c29f2f2cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:12.893566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb3c9d-e5c1-4388-a800-46c29f2f2cf5 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:12.896104 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 194/400] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:12.915138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ed753-9ddc-4ab8-b6fc-1799ccf72144 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:12.915426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ed753-9ddc-4ab8-b6fc-1799ccf72144 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:12.915668 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ed753-9ddc-4ab8-b6fc-1799ccf72144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:12.926255 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361ae05-ccad-4ac7-aed5-d348c1151ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:12.927101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361ae05-ccad-4ac7-aed5-d348c1151ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:12.927432 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361ae05-ccad-4ac7-aed5-d348c1151ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:12.927731 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361ae05-ccad-4ac7-aed5-d348c1151ae6 None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:12.929814 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c361ae05-ccad-4ac7-aed5-d348c1151ae6 None admin] RBAC: Authorizing `identity:get_project(project_id=d659ef9d8a7e447cb15feaa6be1b2787)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:12.932254 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c361ae05-ccad-4ac7-aed5-d348c1151ae6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:12.935530 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:12.945738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 207/401] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:20:12 2022] GET /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787 => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:12.947052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361ae05-ccad-4ac7-aed5-d348c1151ae6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:12.958401 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-bd1ed753-9ddc-4ab8-b6fc-1799ccf72144 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:12.959622 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bd1ed753-9ddc-4ab8-b6fc-1799ccf72144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:13.077415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:13.078501 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 195/402] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:13.081210 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71325614-3d10-4021-97bc-268b66cee095 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:13.082609 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71325614-3d10-4021-97bc-268b66cee095 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:13.083674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71325614-3d10-4021-97bc-268b66cee095 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:13.085808 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 196/403] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:13.092506 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58672748-2add-42c0-9187-c4e7e81471b7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:13.093050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58672748-2add-42c0-9187-c4e7e81471b7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:13.093629 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58672748-2add-42c0-9187-c4e7e81471b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:13.094328 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361ae05-ccad-4ac7-aed5-d348c1151ae6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:13.117305 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-58672748-2add-42c0-9187-c4e7e81471b7 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:13.120593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-58672748-2add-42c0-9187-c4e7e81471b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:13.139995 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3368a-98ad-4e1c-ab93-07384b195eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:13.140699 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3368a-98ad-4e1c-ab93-07384b195eb9 None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:13.140981 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3368a-98ad-4e1c-ab93-07384b195eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:13.141257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3368a-98ad-4e1c-ab93-07384b195eb9 None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787/groups/42a3c71258c541149e49dbe26f21fad1/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:13.143472 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3368a-98ad-4e1c-ab93-07384b195eb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d659ef9d8a7e447cb15feaa6be1b2787, group_id=42a3c71258c541149e49dbe26f21fad1, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:13.145202 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3368a-98ad-4e1c-ab93-07384b195eb9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:13.164916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 208/404] 10.222.0.83 () {58 vars in 1442 bytes} [Mon Aug 1 15:20:12 2022] PUT /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787/groups/42a3c71258c541149e49dbe26f21fad1/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:20:13.174605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3368a-98ad-4e1c-ab93-07384b195eb9 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:13.283613 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 197/405] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:13.352555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58672748-2add-42c0-9187-c4e7e81471b7 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:13.371871 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3368a-98ad-4e1c-ab93-07384b195eb9 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:13.435821 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e357217-8c46-4642-9a3c-f37e6b26a2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:13.438069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e357217-8c46-4642-9a3c-f37e6b26a2fc None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:13.438069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e357217-8c46-4642-9a3c-f37e6b26a2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:13.438069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e357217-8c46-4642-9a3c-f37e6b26a2fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:13.460108 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e357217-8c46-4642-9a3c-f37e6b26a2fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:13.472492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-8e357217-8c46-4642-9a3c-f37e6b26a2fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:20:13.472492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:13.472492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:13.472492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:13.472492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:13.472492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:13.472492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:13.472492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:13.472492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:13.472492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:13.472492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:13.472492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:13.472492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:13.472492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:13.472492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:13.472492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:13.473423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:13.473423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:13.473423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:13.473423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:13.473423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:13.473423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:13.473423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:13.473423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:13.473423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:13.473423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:13.473423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:13.473423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:13.473423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:13.473423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:13.473423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:13.473423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:13.474275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:13.474275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:13.474275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:13.474275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:13.474275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:13.474275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:13.474275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:13.474275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:13.474275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:13.474275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:13.474275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:20:13.474275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:13.474275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 209/406] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:20:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:13.480312 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e357217-8c46-4642-9a3c-f37e6b26a2fc None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:13.504584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:13.566618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:13.662916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58672748-2add-42c0-9187-c4e7e81471b7 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:13.727725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e357217-8c46-4642-9a3c-f37e6b26a2fc None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:13.751598 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5a8e0-f3d8-4cf0-be1f-2d570a3284da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:13.752618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5a8e0-f3d8-4cf0-be1f-2d570a3284da None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:13.753095 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5a8e0-f3d8-4cf0-be1f-2d570a3284da None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:13.753584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5a8e0-f3d8-4cf0-be1f-2d570a3284da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:13.779127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5a8e0-f3d8-4cf0-be1f-2d570a3284da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:13.792036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-65c5a8e0-f3d8-4cf0-be1f-2d570a3284da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:20:13.792036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:13.792036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:13.792036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:13.792036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:13.792036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:13.792036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:13.792036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:13.792036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:13.792036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:13.792036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:13.792036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:13.792036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:13.792036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:13.792036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:13.792036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:13.792996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:13.792996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:13.792996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:13.792996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:13.792996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:13.792996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:13.792996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:13.792996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:13.792996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:13.792996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:13.792996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:13.792996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:13.792996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:13.792996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:13.792996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:13.792996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:13.793806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:13.793806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:13.793806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:13.793806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:13.793806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:13.793806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:13.793806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:13.793806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:13.793806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:13.793806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:13.793806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:20:13.793806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:13.796427 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 198/407] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:20:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 444 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:13.806108 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5a8e0-f3d8-4cf0-be1f-2d570a3284da None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:13.814181 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9c7a1-c138-4258-b505-80197fa9db30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:13.815174 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9c7a1-c138-4258-b505-80197fa9db30 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:13.816163 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9c7a1-c138-4258-b505-80197fa9db30 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:13.821857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9c7a1-c138-4258-b505-80197fa9db30 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:13.823011 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb9c7a1-c138-4258-b505-80197fa9db30 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:13.823514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb9c7a1-c138-4258-b505-80197fa9db30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:13.825217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb9c7a1-c138-4258-b505-80197fa9db30 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:13.845044 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 210/408] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:13.850625 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9c7a1-c138-4258-b505-80197fa9db30 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:14.016139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:14.064654 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5a8e0-f3d8-4cf0-be1f-2d570a3284da None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:14.093120 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9c7a1-c138-4258-b505-80197fa9db30 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:14.122290 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:14.146537 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06788f-9805-4fd9-8e3d-f0d62859ec58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:14.147632 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06788f-9805-4fd9-8e3d-f0d62859ec58 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:14.148167 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06788f-9805-4fd9-8e3d-f0d62859ec58 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:14.148725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06788f-9805-4fd9-8e3d-f0d62859ec58 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:14.151100 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06788f-9805-4fd9-8e3d-f0d62859ec58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:14.151881 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06788f-9805-4fd9-8e3d-f0d62859ec58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:14.156665 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06788f-9805-4fd9-8e3d-f0d62859ec58 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:14.168228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a9dc1-3df2-4c7f-8a31-f02626f8a4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:14.169310 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a9dc1-3df2-4c7f-8a31-f02626f8a4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:14.169839 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a9dc1-3df2-4c7f-8a31-f02626f8a4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:14.170421 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a9dc1-3df2-4c7f-8a31-f02626f8a4b9 None admin] PATH_INFO: `/v3/groups/42a3c71258c541149e49dbe26f21fad1` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:14.172140 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a9dc1-3df2-4c7f-8a31-f02626f8a4b9 None admin] RBAC: Authorizing `identity:get_group(group_id=42a3c71258c541149e49dbe26f21fad1)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:14.173996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a9dc1-3df2-4c7f-8a31-f02626f8a4b9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:14.182272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 199/409] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:14.183546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 211/410] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Aug 1 15:20:13 2022] GET /identity/v3/groups/42a3c71258c541149e49dbe26f21fad1 => generated 226 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:14.186470 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06788f-9805-4fd9-8e3d-f0d62859ec58 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:14.190800 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a9dc1-3df2-4c7f-8a31-f02626f8a4b9 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:14.347645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06788f-9805-4fd9-8e3d-f0d62859ec58 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:14.360033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a9dc1-3df2-4c7f-8a31-f02626f8a4b9 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:14.400476 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba6211-870c-474b-b0af-cbf0f4d8b789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:14.401131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba6211-870c-474b-b0af-cbf0f4d8b789 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:14.401413 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba6211-870c-474b-b0af-cbf0f4d8b789 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:14.401686 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba6211-870c-474b-b0af-cbf0f4d8b789 None admin] PATH_INFO: `/v3/groups/856ad0fafa8d472ca81e7380077787c5` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:14.402887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba6211-870c-474b-b0af-cbf0f4d8b789 None admin] RBAC: Authorizing `identity:get_group(group_id=856ad0fafa8d472ca81e7380077787c5)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:14.404355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba6211-870c-474b-b0af-cbf0f4d8b789 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:14.406319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 200/411] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Aug 1 15:20:14 2022] GET /identity/v3/groups/856ad0fafa8d472ca81e7380077787c5 => generated 229 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:14.406723 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15051be2-5134-43f3-afe4-89a0714373ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:14.407460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e8967-5a3a-4c60-b5fd-6209d298b32b None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:14.407896 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e8967-5a3a-4c60-b5fd-6209d298b32b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:14.408361 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e8967-5a3a-4c60-b5fd-6209d298b32b None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:14.410015 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 201/412] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:14.410943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15051be2-5134-43f3-afe4-89a0714373ed None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:14.411437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15051be2-5134-43f3-afe4-89a0714373ed None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:14.411865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15051be2-5134-43f3-afe4-89a0714373ed None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:14.412747 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e8967-5a3a-4c60-b5fd-6209d298b32b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:14.414778 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15051be2-5134-43f3-afe4-89a0714373ed None admin] RBAC: Authorizing `identity:get_project(project_id=d659ef9d8a7e447cb15feaa6be1b2787)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:14.416265 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15051be2-5134-43f3-afe4-89a0714373ed None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:14.418117 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 212/413] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:20:14 2022] GET /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:14.424927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fddbfe-b3fa-4dd8-a1e2-d65c7348b1ff None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:14.425495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fddbfe-b3fa-4dd8-a1e2-d65c7348b1ff None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:14.425997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fddbfe-b3fa-4dd8-a1e2-d65c7348b1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:14.445809 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f9fddbfe-b3fa-4dd8-a1e2-d65c7348b1ff None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:14.448093 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f9fddbfe-b3fa-4dd8-a1e2-d65c7348b1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:14.451097 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:14.479494 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:14.524227 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 213/414] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:14.525731 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fddbfe-b3fa-4dd8-a1e2-d65c7348b1ff None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:14.531034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e8967-5a3a-4c60-b5fd-6209d298b32b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:14.567937 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927501b9-30a0-4616-9d06-9383ca14cbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:14.568875 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927501b9-30a0-4616-9d06-9383ca14cbcc None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:14.569294 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927501b9-30a0-4616-9d06-9383ca14cbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:14.570019 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927501b9-30a0-4616-9d06-9383ca14cbcc None admin] PATH_INFO: `/v3/projects/6c0cb293301d4dcdbc4412b1e11a284f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:14.571408 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927501b9-30a0-4616-9d06-9383ca14cbcc None admin] RBAC: Authorizing `identity:get_project(project_id=6c0cb293301d4dcdbc4412b1e11a284f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:14.573633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927501b9-30a0-4616-9d06-9383ca14cbcc None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:14.575569 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 202/415] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:20:14 2022] GET /identity/v3/projects/6c0cb293301d4dcdbc4412b1e11a284f => generated 344 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:14.576621 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927501b9-30a0-4616-9d06-9383ca14cbcc None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:14.651060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fddbfe-b3fa-4dd8-a1e2-d65c7348b1ff None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:14.698064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6d3b6-3670-4608-8aa0-f25ddaeaf07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:14.698947 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6d3b6-3670-4608-8aa0-f25ddaeaf07f None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:14.699294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6d3b6-3670-4608-8aa0-f25ddaeaf07f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:14.699637 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6d3b6-3670-4608-8aa0-f25ddaeaf07f None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787/groups/42a3c71258c541149e49dbe26f21fad1/roles/73e29e4f51e6469facf94aaa698c5cfe` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:14.702321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6d3b6-3670-4608-8aa0-f25ddaeaf07f None admin] RBAC: Authorizing `identity:create_grant(project_id=d659ef9d8a7e447cb15feaa6be1b2787, group_id=42a3c71258c541149e49dbe26f21fad1, role_id=73e29e4f51e6469facf94aaa698c5cfe)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:14.703776 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6d3b6-3670-4608-8aa0-f25ddaeaf07f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:14.715530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:14.720080 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927501b9-30a0-4616-9d06-9383ca14cbcc None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:14.724794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 214/416] 10.222.0.83 () {58 vars in 1442 bytes} [Mon Aug 1 15:20:14 2022] PUT /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787/groups/42a3c71258c541149e49dbe26f21fad1/roles/73e29e4f51e6469facf94aaa698c5cfe => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:20:14.725675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6d3b6-3670-4608-8aa0-f25ddaeaf07f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:14.759772 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:14.776247 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769ab9d-12cf-4547-b5c6-35aec06192f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:14.776928 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769ab9d-12cf-4547-b5c6-35aec06192f0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:14.777326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769ab9d-12cf-4547-b5c6-35aec06192f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:14.777686 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769ab9d-12cf-4547-b5c6-35aec06192f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:14.791546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2769ab9d-12cf-4547-b5c6-35aec06192f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:14.798032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-2769ab9d-12cf-4547-b5c6-35aec06192f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:20:14.798032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:14.798032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:14.798032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:14.798032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:14.798032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:14.798032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:14.798032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:14.798032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:14.798032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:14.798032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:14.798032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:14.798032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:14.798032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:14.798032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:14.798032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:14.799037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:14.799037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:14.799037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:14.799037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:14.799037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:14.799037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:14.799037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:14.799037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:14.799037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:14.799037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:14.799037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:14.799037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:14.799037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:14.799037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:14.799037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:14.799037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:14.800507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:14.800507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:14.800507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:14.800507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:14.800507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:14.800507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:14.800507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:14.800507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:14.800507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:14.800507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:14.800507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:20:14.800507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:14.800507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 203/417] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:14.801253 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd4fca-b006-40c9-8b55-b582054ce1d6 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:14.801504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd4fca-b006-40c9-8b55-b582054ce1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:14.801698 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd4fca-b006-40c9-8b55-b582054ce1d6 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:14.803545 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 204/418] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:14.805254 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd4fca-b006-40c9-8b55-b582054ce1d6 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:14.858912 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6d3b6-3670-4608-8aa0-f25ddaeaf07f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:14.901306 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a68a26-9312-40ad-945f-5237b4bb291b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:14.901915 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a68a26-9312-40ad-945f-5237b4bb291b None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:14.902222 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a68a26-9312-40ad-945f-5237b4bb291b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:14.902505 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a68a26-9312-40ad-945f-5237b4bb291b None admin] PATH_INFO: `/v3/projects/6c0cb293301d4dcdbc4412b1e11a284f/groups/856ad0fafa8d472ca81e7380077787c5/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:14.904668 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a68a26-9312-40ad-945f-5237b4bb291b None admin] RBAC: Authorizing `identity:create_grant(project_id=6c0cb293301d4dcdbc4412b1e11a284f, group_id=856ad0fafa8d472ca81e7380077787c5, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:14.906212 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a68a26-9312-40ad-945f-5237b4bb291b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:14.922010 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 215/419] 10.222.0.83 () {58 vars in 1442 bytes} [Mon Aug 1 15:20:14 2022] PUT /identity/v3/projects/6c0cb293301d4dcdbc4412b1e11a284f/groups/856ad0fafa8d472ca81e7380077787c5/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:20:14.923444 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91775d44-e767-48b8-bb4c-2511cea61fea None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:14.923780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91775d44-e767-48b8-bb4c-2511cea61fea None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:14.924058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91775d44-e767-48b8-bb4c-2511cea61fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:14.946240 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-91775d44-e767-48b8-bb4c-2511cea61fea None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:14.947242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-91775d44-e767-48b8-bb4c-2511cea61fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:14.952209 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd4fca-b006-40c9-8b55-b582054ce1d6 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:15.011732 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b52996-6ad1-4718-b374-f4c5e415020e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:15.018026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b52996-6ad1-4718-b374-f4c5e415020e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:15.018599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b52996-6ad1-4718-b374-f4c5e415020e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:15.018953 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b52996-6ad1-4718-b374-f4c5e415020e None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:15.019890 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b52996-6ad1-4718-b374-f4c5e415020e None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:15.020302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b52996-6ad1-4718-b374-f4c5e415020e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:15.022114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b52996-6ad1-4718-b374-f4c5e415020e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:15.044016 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 205/420] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:20:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:15.053396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b52996-6ad1-4718-b374-f4c5e415020e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:15.062499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:15.089508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 216/421] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:15.113353 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91775d44-e767-48b8-bb4c-2511cea61fea None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:15.156734 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:15.234687 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b52996-6ad1-4718-b374-f4c5e415020e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:15.272433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f748a50-dd18-4631-9b50-6b71149a2db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:15.277918 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f748a50-dd18-4631-9b50-6b71149a2db5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:15.277918 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f748a50-dd18-4631-9b50-6b71149a2db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:15.277918 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f748a50-dd18-4631-9b50-6b71149a2db5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:15.280246 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91775d44-e767-48b8-bb4c-2511cea61fea None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:15.355866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a69295-68f0-443d-9894-80fb77069a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:15.356529 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a69295-68f0-443d-9894-80fb77069a39 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:15.356815 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a69295-68f0-443d-9894-80fb77069a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:15.357087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a69295-68f0-443d-9894-80fb77069a39 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:15.377239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f748a50-dd18-4631-9b50-6b71149a2db5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:15.378873 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a69295-68f0-443d-9894-80fb77069a39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:15.385339 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f748a50-dd18-4631-9b50-6b71149a2db5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:15.393627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-c0a69295-68f0-443d-9894-80fb77069a39 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:20:15.393627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:15.393627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:15.393627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:15.393627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:15.393627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:15.393627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:15.393627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:15.393627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:15.393627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:15.393627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:15.393627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:15.393627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:15.393627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:15.393627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:15.393627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:15.394856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:15.394856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:15.394856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:15.394856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:15.394856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:15.394856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:15.394856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:15.394856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:15.394856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:15.394856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:15.394856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:15.394856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:15.394856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:15.394856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:15.394856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:15.394856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:15.396350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:15.396350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:15.396350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:15.396350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:15.396350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:15.396350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:15.396350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:15.396350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:15.396350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:15.396350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:15.396350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:20:15.396350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:15.396350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 217/422] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:20:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:20:15.404347 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a69295-68f0-443d-9894-80fb77069a39 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:15.494690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:15.528710 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:15.630238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 206/423] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:15.638455 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f748a50-dd18-4631-9b50-6b71149a2db5 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:15.653875 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a69295-68f0-443d-9894-80fb77069a39 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:15.745018 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ff395-1ecc-4839-895f-2a1f44c12ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:15.745765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ff395-1ecc-4839-895f-2a1f44c12ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:15.746311 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ff395-1ecc-4839-895f-2a1f44c12ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:15.746668 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ff395-1ecc-4839-895f-2a1f44c12ec6 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:15.747594 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ff395-1ecc-4839-895f-2a1f44c12ec6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:15.748172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ff395-1ecc-4839-895f-2a1f44c12ec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:15.749963 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ff395-1ecc-4839-895f-2a1f44c12ec6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:15.762276 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 218/424] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:15.768100 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ff395-1ecc-4839-895f-2a1f44c12ec6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:15.897378 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f748a50-dd18-4631-9b50-6b71149a2db5 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:15.911607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ff395-1ecc-4839-895f-2a1f44c12ec6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:15.926652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:15.946770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24ec1e-3b60-4c2d-ad39-e994f15ab14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:15.948324 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24ec1e-3b60-4c2d-ad39-e994f15ab14e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:15.948781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24ec1e-3b60-4c2d-ad39-e994f15ab14e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:15.949258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24ec1e-3b60-4c2d-ad39-e994f15ab14e None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:15.950679 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24ec1e-3b60-4c2d-ad39-e994f15ab14e None admin] RBAC: Authorizing `identity:get_project(project_id=44e0498d623545e68ea89aa6de0a8be7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:15.952392 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24ec1e-3b60-4c2d-ad39-e994f15ab14e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:15.958008 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 207/425] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:20:15 2022] GET /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7 => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:15.959236 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dfc01-0cf2-440d-9767-3b0feca8ad48 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:15.959365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dfc01-0cf2-440d-9767-3b0feca8ad48 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:15.959768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dfc01-0cf2-440d-9767-3b0feca8ad48 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:15.961038 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 208/426] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:15.963920 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2dfc01-0cf2-440d-9767-3b0feca8ad48 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:15.967815 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8f488-d410-4376-b780-395b1d02150c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:15.968452 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8f488-d410-4376-b780-395b1d02150c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:15.968731 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8f488-d410-4376-b780-395b1d02150c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:15.969000 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8f488-d410-4376-b780-395b1d02150c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:15.982322 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:16.042924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8f488-d410-4376-b780-395b1d02150c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:16.044278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8f488-d410-4376-b780-395b1d02150c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:16.098109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2dfc01-0cf2-440d-9767-3b0feca8ad48 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:16.143584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1309d19-ff50-4576-8d73-129843a1ca20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:16.144764 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1309d19-ff50-4576-8d73-129843a1ca20 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:16.145362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1309d19-ff50-4576-8d73-129843a1ca20 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:16.145965 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1309d19-ff50-4576-8d73-129843a1ca20 None admin] PATH_INFO: `/v3/groups/42a3c71258c541149e49dbe26f21fad1` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:16.147447 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1309d19-ff50-4576-8d73-129843a1ca20 None admin] RBAC: Authorizing `identity:get_group(group_id=42a3c71258c541149e49dbe26f21fad1)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:16.149225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1309d19-ff50-4576-8d73-129843a1ca20 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:16.152081 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 209/427] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Aug 1 15:20:15 2022] GET /identity/v3/groups/42a3c71258c541149e49dbe26f21fad1 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:16.153649 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32bcba-a5bd-47ac-9640-58fa0b8a432f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:16.154227 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32bcba-a5bd-47ac-9640-58fa0b8a432f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:16.154743 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32bcba-a5bd-47ac-9640-58fa0b8a432f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:16.162878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 219/428] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:16.164093 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8f488-d410-4376-b780-395b1d02150c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:16.176502 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-1d32bcba-a5bd-47ac-9640-58fa0b8a432f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:16.178717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:16.180386 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1d32bcba-a5bd-47ac-9640-58fa0b8a432f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:16.261145 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 210/429] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:16.262402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32bcba-a5bd-47ac-9640-58fa0b8a432f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:16.278644 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:16.306836 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8f488-d410-4376-b780-395b1d02150c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:16.354304 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab97df-3802-4155-b1d8-dbb3c4f7d560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:16.355081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab97df-3802-4155-b1d8-dbb3c4f7d560 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:16.355367 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab97df-3802-4155-b1d8-dbb3c4f7d560 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:16.355645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab97df-3802-4155-b1d8-dbb3c4f7d560 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:16.356463 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab97df-3802-4155-b1d8-dbb3c4f7d560 None admin] RBAC: Adding query filter params (group.id=42a3c71258c541149e49dbe26f21fad1, role.id=9cda4e2827dc4c2e87477bef8f319f92, scope.project.id=44e0498d623545e68ea89aa6de0a8be7) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:16.356773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab97df-3802-4155-b1d8-dbb3c4f7d560 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:16.359100 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab97df-3802-4155-b1d8-dbb3c4f7d560 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:16.366566 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 220/430] 10.222.0.83 () {56 vars in 1376 bytes} [Mon Aug 1 15:20:16 2022] GET /identity/v3/role_assignments?group.id=42a3c71258c541149e49dbe26f21fad1&scope.project.id=44e0498d623545e68ea89aa6de0a8be7&role.id=9cda4e2827dc4c2e87477bef8f319f92 => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:16.367821 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab97df-3802-4155-b1d8-dbb3c4f7d560 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:16.383608 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32bcba-a5bd-47ac-9640-58fa0b8a432f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:16.432692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6759d32-a864-452a-9f7e-0cd28e0ffedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:16.433492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6759d32-a864-452a-9f7e-0cd28e0ffedd None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:16.433786 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6759d32-a864-452a-9f7e-0cd28e0ffedd None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:16.434100 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6759d32-a864-452a-9f7e-0cd28e0ffedd None admin] PATH_INFO: `/v3/projects/44e0498d623545e68ea89aa6de0a8be7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:16.435726 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6759d32-a864-452a-9f7e-0cd28e0ffedd None admin] RBAC: Authorizing `identity:get_project(project_id=44e0498d623545e68ea89aa6de0a8be7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:16.437245 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6759d32-a864-452a-9f7e-0cd28e0ffedd None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:16.439966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 211/431] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:20:16 2022] GET /identity/v3/projects/44e0498d623545e68ea89aa6de0a8be7 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:16.448293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6759d32-a864-452a-9f7e-0cd28e0ffedd None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:16.492345 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:16.553564 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab97df-3802-4155-b1d8-dbb3c4f7d560 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:16.609546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:16.610885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e55718-8131-4674-a421-dd317306c8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:16.611723 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e55718-8131-4674-a421-dd317306c8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:16.612180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e55718-8131-4674-a421-dd317306c8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:16.612736 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e55718-8131-4674-a421-dd317306c8d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:16.626613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e55718-8131-4674-a421-dd317306c8d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:16.633296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-15e55718-8131-4674-a421-dd317306c8d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:20:16.633296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:16.633296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:16.633296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:16.633296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:16.633296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:16.633296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:16.633296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:16.633296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:16.633296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:16.633296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:16.633296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:16.633296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:16.633296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:16.633296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:16.633296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:16.634322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:16.634322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:16.634322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:16.634322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:16.634322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:16.634322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:16.634322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:16.634322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:16.634322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:16.634322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:16.634322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:16.634322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:16.634322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:16.634322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:16.634322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:16.634322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:16.635715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:16.635715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:16.635715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:16.635715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:16.635715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:16.635715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:16.635715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:16.635715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:16.635715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:16.635715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:16.635715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:20:16.635715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:16.636453 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6759d32-a864-452a-9f7e-0cd28e0ffedd None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:16.638972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 221/432] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:16.647503 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e55718-8131-4674-a421-dd317306c8d1 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:16.691470 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53cee60-f61c-4d60-b648-7f08aebc94f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:16.693278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53cee60-f61c-4d60-b648-7f08aebc94f9 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:16.694198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53cee60-f61c-4d60-b648-7f08aebc94f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:16.695178 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53cee60-f61c-4d60-b648-7f08aebc94f9 None admin] PATH_INFO: `/v3/groups/42a3c71258c541149e49dbe26f21fad1` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:16.699942 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53cee60-f61c-4d60-b648-7f08aebc94f9 None admin] RBAC: Authorizing `identity:get_group(group_id=42a3c71258c541149e49dbe26f21fad1)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:16.702997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53cee60-f61c-4d60-b648-7f08aebc94f9 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:16.707530 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 212/433] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Aug 1 15:20:16 2022] GET /identity/v3/groups/42a3c71258c541149e49dbe26f21fad1 => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:16.715972 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53cee60-f61c-4d60-b648-7f08aebc94f9 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:16.793718 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e55718-8131-4674-a421-dd317306c8d1 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:16.840379 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6718076-c2f0-437a-a506-aab84bb9a007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:16.841341 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6718076-c2f0-437a-a506-aab84bb9a007 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:16.841817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6718076-c2f0-437a-a506-aab84bb9a007 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:16.842329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6718076-c2f0-437a-a506-aab84bb9a007 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:16.843234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6718076-c2f0-437a-a506-aab84bb9a007 None admin] RBAC: Adding query filter params (name=member) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:16.844309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6718076-c2f0-437a-a506-aab84bb9a007 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:16.846348 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6718076-c2f0-437a-a506-aab84bb9a007 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:16.850229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:16.859416 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53cee60-f61c-4d60-b648-7f08aebc94f9 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:16.859658 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 222/434] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:20:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:16.868172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6718076-c2f0-437a-a506-aab84bb9a007 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:16.910616 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472b10b-34d9-4d6e-bb7b-aeffc27ef26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:16.911242 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472b10b-34d9-4d6e-bb7b-aeffc27ef26f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:16.911521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472b10b-34d9-4d6e-bb7b-aeffc27ef26f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:16.911793 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472b10b-34d9-4d6e-bb7b-aeffc27ef26f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:16.912596 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472b10b-34d9-4d6e-bb7b-aeffc27ef26f None admin] RBAC: Adding query filter params (group.id=42a3c71258c541149e49dbe26f21fad1, role.id=73e29e4f51e6469facf94aaa698c5cfe, scope.project.id=44e0498d623545e68ea89aa6de0a8be7) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:16.912903 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472b10b-34d9-4d6e-bb7b-aeffc27ef26f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:16.914413 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472b10b-34d9-4d6e-bb7b-aeffc27ef26f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:16.923676 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 213/435] 10.222.0.83 () {56 vars in 1376 bytes} [Mon Aug 1 15:20:16 2022] GET /identity/v3/role_assignments?group.id=42a3c71258c541149e49dbe26f21fad1&scope.project.id=44e0498d623545e68ea89aa6de0a8be7&role.id=73e29e4f51e6469facf94aaa698c5cfe => generated 611 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:17.013490 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6718076-c2f0-437a-a506-aab84bb9a007 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:17.070550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd05824d-2ed3-4f74-82b6-537ce41e1598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:17.071434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05824d-2ed3-4f74-82b6-537ce41e1598 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:17.071773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05824d-2ed3-4f74-82b6-537ce41e1598 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:17.072167 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05824d-2ed3-4f74-82b6-537ce41e1598 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:17.158799 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:17.168253 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd05824d-2ed3-4f74-82b6-537ce41e1598 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:17.169722 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd05824d-2ed3-4f74-82b6-537ce41e1598 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:17.301972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 223/436] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:17.310287 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472b10b-34d9-4d6e-bb7b-aeffc27ef26f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:17.328686 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:17.332927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1763e2-ed34-4c0c-a025-55e75f898b41 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:17.333282 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1763e2-ed34-4c0c-a025-55e75f898b41 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:17.333618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1763e2-ed34-4c0c-a025-55e75f898b41 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:17.336556 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 224/437] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:17.343063 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11984bea-bfed-4c85-90b6-ed0793abaf0a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:17.344174 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11984bea-bfed-4c85-90b6-ed0793abaf0a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:17.344522 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11984bea-bfed-4c85-90b6-ed0793abaf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:17.374660 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-11984bea-bfed-4c85-90b6-ed0793abaf0a None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:17.376251 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-11984bea-bfed-4c85-90b6-ed0793abaf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:17.459985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472b10b-34d9-4d6e-bb7b-aeffc27ef26f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:17.497246 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 225/438] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:20:17.510287 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e9b26-4a4c-4147-8747-00363cdb06fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:17.510946 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e9b26-4a4c-4147-8747-00363cdb06fa None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:17.511569 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e9b26-4a4c-4147-8747-00363cdb06fa None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:17.511569 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e9b26-4a4c-4147-8747-00363cdb06fa None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:17.513393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e9b26-4a4c-4147-8747-00363cdb06fa None admin] RBAC: Authorizing `identity:get_project(project_id=d659ef9d8a7e447cb15feaa6be1b2787)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:17.515192 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e9b26-4a4c-4147-8747-00363cdb06fa None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:17.517400 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 214/439] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:20:17 2022] GET /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:17.526302 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11984bea-bfed-4c85-90b6-ed0793abaf0a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:17.533473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e9b26-4a4c-4147-8747-00363cdb06fa None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:17.575759 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:17.639568 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:17.663182 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11984bea-bfed-4c85-90b6-ed0793abaf0a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:17.664348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e9b26-4a4c-4147-8747-00363cdb06fa None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:17.700540 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d16f6-6983-4f63-8708-154dcb2a6aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:17.701174 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d16f6-6983-4f63-8708-154dcb2a6aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:17.701460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d16f6-6983-4f63-8708-154dcb2a6aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:17.701737 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d16f6-6983-4f63-8708-154dcb2a6aa1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:17.703377 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43a012-55cc-4fe0-b1c1-f59a1f561e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:17.704021 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43a012-55cc-4fe0-b1c1-f59a1f561e0c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:17.704302 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43a012-55cc-4fe0-b1c1-f59a1f561e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:17.704580 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43a012-55cc-4fe0-b1c1-f59a1f561e0c None admin] PATH_INFO: `/v3/groups/42a3c71258c541149e49dbe26f21fad1` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:17.705784 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43a012-55cc-4fe0-b1c1-f59a1f561e0c None admin] RBAC: Authorizing `identity:get_group(group_id=42a3c71258c541149e49dbe26f21fad1)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:17.707401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43a012-55cc-4fe0-b1c1-f59a1f561e0c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:17.709467 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 226/440] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Aug 1 15:20:17 2022] GET /identity/v3/groups/42a3c71258c541149e49dbe26f21fad1 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:17.710904 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d81f0b-320e-4229-961f-563106b6fe45 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:17.711066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d81f0b-320e-4229-961f-563106b6fe45 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:17.711371 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d81f0b-320e-4229-961f-563106b6fe45 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:17.713364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 227/441] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:17.718158 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d81f0b-320e-4229-961f-563106b6fe45 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:17.722086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d16f6-6983-4f63-8708-154dcb2a6aa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:17.729370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-b36d16f6-6983-4f63-8708-154dcb2a6aa1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:20:17.729370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:17.729370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:17.729370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:17.729370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:17.729370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:17.729370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:17.729370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:17.729370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:17.729370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:17.729370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:17.729370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:17.729370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:17.729370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:17.729370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:17.729370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:17.730866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:17.730866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:17.730866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:17.730866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:17.730866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:17.730866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:17.730866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:17.730866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:17.730866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:17.730866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:17.730866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:17.730866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:17.730866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:17.730866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:17.730866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:17.730866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:17.733312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:17.733312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:17.733312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:17.733312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:17.733312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:17.733312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:17.733312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:17.733312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:17.733312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:17.733312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:17.733312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:20:17.733312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:17.733312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 215/442] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:20:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:17.733312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda1f40-1a6c-457e-b1df-4d8cd9d6a846 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:17.735559 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda1f40-1a6c-457e-b1df-4d8cd9d6a846 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:17.735559 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda1f40-1a6c-457e-b1df-4d8cd9d6a846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:17.752667 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-bfda1f40-1a6c-457e-b1df-4d8cd9d6a846 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:17.754139 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bfda1f40-1a6c-457e-b1df-4d8cd9d6a846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:17.828934 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:17.831461 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 216/443] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:17.832310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda1f40-1a6c-457e-b1df-4d8cd9d6a846 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:17.839746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d81f0b-320e-4229-961f-563106b6fe45 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:17.883726 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50343182-0b60-47a5-b7a9-b2256145f757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:17.884355 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50343182-0b60-47a5-b7a9-b2256145f757 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:17.885125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50343182-0b60-47a5-b7a9-b2256145f757 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:17.885391 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50343182-0b60-47a5-b7a9-b2256145f757 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:17.886211 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50343182-0b60-47a5-b7a9-b2256145f757 None admin] RBAC: Adding query filter params (group.id=42a3c71258c541149e49dbe26f21fad1, role.id=9cda4e2827dc4c2e87477bef8f319f92, scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:17.886520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50343182-0b60-47a5-b7a9-b2256145f757 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:17.887971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50343182-0b60-47a5-b7a9-b2256145f757 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:17.897501 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 228/444] 10.222.0.83 () {56 vars in 1376 bytes} [Mon Aug 1 15:20:17 2022] GET /identity/v3/role_assignments?group.id=42a3c71258c541149e49dbe26f21fad1&scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787&role.id=9cda4e2827dc4c2e87477bef8f319f92 => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:17.903223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50343182-0b60-47a5-b7a9-b2256145f757 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:17.905388 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:17.950850 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda1f40-1a6c-457e-b1df-4d8cd9d6a846 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:17.992191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d4850-a8b2-4792-b946-d0a0fec7a9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:17.992738 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d4850-a8b2-4792-b946-d0a0fec7a9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:17.993016 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d4850-a8b2-4792-b946-d0a0fec7a9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:17.993289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d4850-a8b2-4792-b946-d0a0fec7a9a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:17.994065 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d4850-a8b2-4792-b946-d0a0fec7a9a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:17.994374 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d4850-a8b2-4792-b946-d0a0fec7a9a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:17.995969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d4850-a8b2-4792-b946-d0a0fec7a9a7 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:18.006195 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 217/445] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:18.012397 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d4850-a8b2-4792-b946-d0a0fec7a9a7 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:18.031829 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50343182-0b60-47a5-b7a9-b2256145f757 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:18.079043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149791e0-fba4-4c71-be14-304f118ae89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:18.079883 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149791e0-fba4-4c71-be14-304f118ae89d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:18.080174 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149791e0-fba4-4c71-be14-304f118ae89d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:18.080489 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149791e0-fba4-4c71-be14-304f118ae89d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:18.097160 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149791e0-fba4-4c71-be14-304f118ae89d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:18.104356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-149791e0-fba4-4c71-be14-304f118ae89d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:20:18.104356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:18.104356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:18.104356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:18.104356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:18.104356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:18.104356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:18.104356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:18.104356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:18.104356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:18.104356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:18.104356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:18.104356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:18.104356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:18.104356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:18.104356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:18.105870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:18.105870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:18.105870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:18.105870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:18.105870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:18.105870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:18.105870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:18.105870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:18.105870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:18.105870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:18.105870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:18.105870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:18.105870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:18.105870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:18.105870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:18.105870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:18.106950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:18.106950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:18.106950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:18.106950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:18.106950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:18.106950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:18.106950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:18.106950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:18.106950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:18.106950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:18.106950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:20:18.106950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:18.106950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 229/446] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:20:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:18.117651 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149791e0-fba4-4c71-be14-304f118ae89d None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:18.142900 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:18.163485 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d4850-a8b2-4792-b946-d0a0fec7a9a7 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:18.209751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:18.222224 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1600c5-90cf-49e9-b6b7-4c0ee8b52954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:18.222971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1600c5-90cf-49e9-b6b7-4c0ee8b52954 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:18.223242 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1600c5-90cf-49e9-b6b7-4c0ee8b52954 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:18.223439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1600c5-90cf-49e9-b6b7-4c0ee8b52954 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:18.251972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149791e0-fba4-4c71-be14-304f118ae89d None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:18.297638 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768b0ac-7a4b-4779-ac1d-9ff8e805b7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:18.298237 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768b0ac-7a4b-4779-ac1d-9ff8e805b7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:18.298522 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768b0ac-7a4b-4779-ac1d-9ff8e805b7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:18.298795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768b0ac-7a4b-4779-ac1d-9ff8e805b7d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:18.299544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768b0ac-7a4b-4779-ac1d-9ff8e805b7d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:18.299823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768b0ac-7a4b-4779-ac1d-9ff8e805b7d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:18.301213 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768b0ac-7a4b-4779-ac1d-9ff8e805b7d6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:18.315173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 230/447] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:18.315351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1600c5-90cf-49e9-b6b7-4c0ee8b52954 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:18.315351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1600c5-90cf-49e9-b6b7-4c0ee8b52954 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:18.316314 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768b0ac-7a4b-4779-ac1d-9ff8e805b7d6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:18.398372 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:18.417456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 218/448] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 15:20:18.418825 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768b0ac-7a4b-4779-ac1d-9ff8e805b7d6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:18.423228 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1600c5-90cf-49e9-b6b7-4c0ee8b52954 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:18.452997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f45617-538c-407c-855b-8f98257d8ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:18.453250 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:18.453807 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f45617-538c-407c-855b-8f98257d8ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:18.453946 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f45617-538c-407c-855b-8f98257d8ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:18.454189 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f45617-538c-407c-855b-8f98257d8ec7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:18.522613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f45617-538c-407c-855b-8f98257d8ec7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:18.523975 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f45617-538c-407c-855b-8f98257d8ec7 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:18.526625 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1600c5-90cf-49e9-b6b7-4c0ee8b52954 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:18.565451 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc7fff-e0da-4565-b1e5-37bc1c603284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:18.566177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc7fff-e0da-4565-b1e5-37bc1c603284 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:18.566177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc7fff-e0da-4565-b1e5-37bc1c603284 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:18.566847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc7fff-e0da-4565-b1e5-37bc1c603284 None admin] PATH_INFO: `/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:18.567987 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc7fff-e0da-4565-b1e5-37bc1c603284 None admin] RBAC: Authorizing `identity:get_project(project_id=d659ef9d8a7e447cb15feaa6be1b2787)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:18.569438 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc7fff-e0da-4565-b1e5-37bc1c603284 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:18.571533 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 219/449] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:20:18 2022] GET /identity/v3/projects/d659ef9d8a7e447cb15feaa6be1b2787 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:18.576958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc7fff-e0da-4565-b1e5-37bc1c603284 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:18.608891 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:18.629953 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 231/450] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 15:20:18.636042 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f45617-538c-407c-855b-8f98257d8ec7 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:18.684415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:18.689288 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc7fff-e0da-4565-b1e5-37bc1c603284 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:18.733747 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d91653-93cc-4b39-b4de-119b07a360f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:18.734643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d91653-93cc-4b39-b4de-119b07a360f4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:18.735022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d91653-93cc-4b39-b4de-119b07a360f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:18.735510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d91653-93cc-4b39-b4de-119b07a360f4 None admin] PATH_INFO: `/v3/groups/42a3c71258c541149e49dbe26f21fad1` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:18.737191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d91653-93cc-4b39-b4de-119b07a360f4 None admin] RBAC: Authorizing `identity:get_group(group_id=42a3c71258c541149e49dbe26f21fad1)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:18.739079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d91653-93cc-4b39-b4de-119b07a360f4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:18.747125 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 220/451] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Aug 1 15:20:18 2022] GET /identity/v3/groups/42a3c71258c541149e49dbe26f21fad1 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:18.747322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f45617-538c-407c-855b-8f98257d8ec7 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:18.748380 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d91653-93cc-4b39-b4de-119b07a360f4 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:18.776700 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5d413-d91f-4a89-9c8e-0530d0271227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:18.777264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5d413-d91f-4a89-9c8e-0530d0271227 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:18.777545 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5d413-d91f-4a89-9c8e-0530d0271227 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:18.777816 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5d413-d91f-4a89-9c8e-0530d0271227 None admin] PATH_INFO: `/v3/projects/6c0cb293301d4dcdbc4412b1e11a284f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:18.779016 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5d413-d91f-4a89-9c8e-0530d0271227 None admin] RBAC: Authorizing `identity:get_project(project_id=6c0cb293301d4dcdbc4412b1e11a284f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:18.780450 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5d413-d91f-4a89-9c8e-0530d0271227 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:18.782158 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 232/452] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:20:18 2022] GET /identity/v3/projects/6c0cb293301d4dcdbc4412b1e11a284f => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:18.787572 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5d413-d91f-4a89-9c8e-0530d0271227 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:18.836910 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:18.857579 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d91653-93cc-4b39-b4de-119b07a360f4 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:18.893968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5d413-d91f-4a89-9c8e-0530d0271227 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:18.894353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a90b3f-5fd2-41db-ac69-65604c57e515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:18.895039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a90b3f-5fd2-41db-ac69-65604c57e515 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:18.895310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a90b3f-5fd2-41db-ac69-65604c57e515 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:18.895544 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a90b3f-5fd2-41db-ac69-65604c57e515 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:18.896350 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a90b3f-5fd2-41db-ac69-65604c57e515 None admin] RBAC: Adding query filter params (group.id=42a3c71258c541149e49dbe26f21fad1, role.id=73e29e4f51e6469facf94aaa698c5cfe, scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:18.896664 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a90b3f-5fd2-41db-ac69-65604c57e515 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:18.898211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a90b3f-5fd2-41db-ac69-65604c57e515 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:18.905113 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 221/453] 10.222.0.83 () {56 vars in 1376 bytes} [Mon Aug 1 15:20:18 2022] GET /identity/v3/role_assignments?group.id=42a3c71258c541149e49dbe26f21fad1&scope.project.id=d659ef9d8a7e447cb15feaa6be1b2787&role.id=73e29e4f51e6469facf94aaa698c5cfe => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:18.938732 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25aae11-90d5-4732-84a9-ac39e6bdbfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:18.939387 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25aae11-90d5-4732-84a9-ac39e6bdbfcd None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:18.939671 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25aae11-90d5-4732-84a9-ac39e6bdbfcd None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:18.939949 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25aae11-90d5-4732-84a9-ac39e6bdbfcd None admin] PATH_INFO: `/v3/groups/856ad0fafa8d472ca81e7380077787c5` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:18.941204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25aae11-90d5-4732-84a9-ac39e6bdbfcd None admin] RBAC: Authorizing `identity:get_group(group_id=856ad0fafa8d472ca81e7380077787c5)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:18.942745 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25aae11-90d5-4732-84a9-ac39e6bdbfcd None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:18.944439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 233/454] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Aug 1 15:20:18 2022] GET /identity/v3/groups/856ad0fafa8d472ca81e7380077787c5 => generated 229 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:18.951173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a90b3f-5fd2-41db-ac69-65604c57e515 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:18.988719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:19.066835 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a90b3f-5fd2-41db-ac69-65604c57e515 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:19.104914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb5dee-3bd2-42bf-87bb-278a8344c288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:19.105554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb5dee-3bd2-42bf-87bb-278a8344c288 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:19.105832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb5dee-3bd2-42bf-87bb-278a8344c288 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:19.106158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb5dee-3bd2-42bf-87bb-278a8344c288 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:19.106949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb5dee-3bd2-42bf-87bb-278a8344c288 None admin] RBAC: Adding query filter params (group.id=856ad0fafa8d472ca81e7380077787c5, role.id=7bdfa43dc1d046f7965ffe52b74bc58f, scope.project.id=6c0cb293301d4dcdbc4412b1e11a284f) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:19.107268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb5dee-3bd2-42bf-87bb-278a8344c288 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:19.109387 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb5dee-3bd2-42bf-87bb-278a8344c288 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:19.116699 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 222/455] 10.222.0.83 () {56 vars in 1376 bytes} [Mon Aug 1 15:20:18 2022] GET /identity/v3/role_assignments?group.id=856ad0fafa8d472ca81e7380077787c5&scope.project.id=6c0cb293301d4dcdbc4412b1e11a284f&role.id=7bdfa43dc1d046f7965ffe52b74bc58f => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:22.079791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ea2e2-7a11-45b5-9566-f28d97f691cc None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:22.080761 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ea2e2-7a11-45b5-9566-f28d97f691cc None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:22.081243 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ea2e2-7a11-45b5-9566-f28d97f691cc None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:22.083178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 234/456] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:22.089495 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22927574-6f9d-4374-a585-cd9c5ceb7afd None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:22.090511 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22927574-6f9d-4374-a585-cd9c5ceb7afd None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:22.090993 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22927574-6f9d-4374-a585-cd9c5ceb7afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:22.118973 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-22927574-6f9d-4374-a585-cd9c5ceb7afd None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:22.120575 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-22927574-6f9d-4374-a585-cd9c5ceb7afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:22.226804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 223/457] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:22.269540 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22927574-6f9d-4374-a585-cd9c5ceb7afd None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:22.269867 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b76a08-8997-4087-ac3b-46234f44c09e None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:22.270155 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b76a08-8997-4087-ac3b-46234f44c09e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:22.270706 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b76a08-8997-4087-ac3b-46234f44c09e None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:22.272322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 235/458] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:22.279761 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06feedb7-0e81-483c-bc72-89959ba4c203 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:22.280371 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06feedb7-0e81-483c-bc72-89959ba4c203 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:22.280651 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06feedb7-0e81-483c-bc72-89959ba4c203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:22.302360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-06feedb7-0e81-483c-bc72-89959ba4c203 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:22.304031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-06feedb7-0e81-483c-bc72-89959ba4c203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:22.316576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:22.397985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 236/459] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:20:22.399733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798e138-bc2e-4327-b9a8-388caa56239c None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:22.400872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798e138-bc2e-4327-b9a8-388caa56239c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:22.401380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798e138-bc2e-4327-b9a8-388caa56239c None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:22.406399 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 237/460] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:22.410752 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a641f-6d3c-4d38-afee-321d358acf4c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:22.412109 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a641f-6d3c-4d38-afee-321d358acf4c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:22.412576 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a641f-6d3c-4d38-afee-321d358acf4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:22.444695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-258a641f-6d3c-4d38-afee-321d358acf4c None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:22.446619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-258a641f-6d3c-4d38-afee-321d358acf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:22.450976 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22927574-6f9d-4374-a585-cd9c5ceb7afd None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:22.497730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44448feb-a0be-4099-b727-4868f28ba23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:22.499259 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44448feb-a0be-4099-b727-4868f28ba23a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:22.500020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44448feb-a0be-4099-b727-4868f28ba23a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:22.500798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44448feb-a0be-4099-b727-4868f28ba23a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:22.517244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44448feb-a0be-4099-b727-4868f28ba23a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:22.519050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44448feb-a0be-4099-b727-4868f28ba23a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:22.530175 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:22.533248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 238/461] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:20:22.534361 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a641f-6d3c-4d38-afee-321d358acf4c None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:22.538267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-44448feb-a0be-4099-b727-4868f28ba23a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:22.538267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:22.538267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:22.538267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:22.538267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:22.538267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:22.538267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:22.538267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:22.538267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:22.538267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:22.538267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:22.538267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:22.538267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:22.538267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:22.538267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:22.539122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:22.539122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:22.539122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:22.539122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:22.539122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:22.539122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:22.539122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:22.539122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:22.539122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:22.539122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:22.539122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:22.539122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:22.539122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:22.539122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:22.539122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:22.539122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:22.539122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:22.540076 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:22.540076 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:22.540076 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:22.540076 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:22.542355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 224/462] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:22.551226 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44448feb-a0be-4099-b727-4868f28ba23a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:22.657645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:22.661196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a641f-6d3c-4d38-afee-321d358acf4c None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:22.687839 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44448feb-a0be-4099-b727-4868f28ba23a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:22.702165 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a891a-7794-4fed-bfa2-a52f94e077cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:22.703175 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a891a-7794-4fed-bfa2-a52f94e077cc None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:22.703599 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a891a-7794-4fed-bfa2-a52f94e077cc None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:22.703874 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a891a-7794-4fed-bfa2-a52f94e077cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:22.714805 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a891a-7794-4fed-bfa2-a52f94e077cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:22.716249 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a891a-7794-4fed-bfa2-a52f94e077cc None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:22.724246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9cecd5-addd-4a2a-9f3e-a9f90375a959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:22.724896 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9cecd5-addd-4a2a-9f3e-a9f90375a959 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:22.725582 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9cecd5-addd-4a2a-9f3e-a9f90375a959 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:22.725864 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9cecd5-addd-4a2a-9f3e-a9f90375a959 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:22.726595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9cecd5-addd-4a2a-9f3e-a9f90375a959 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:22.727014 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9cecd5-addd-4a2a-9f3e-a9f90375a959 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:22.728314 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9cecd5-addd-4a2a-9f3e-a9f90375a959 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:22.731725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-868a891a-7794-4fed-bfa2-a52f94e077cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:22.731725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:22.731725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:22.731725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:22.731725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:22.731725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:22.731725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:22.731725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:22.731725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:22.731725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:22.731725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:22.731725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:22.731725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:22.731725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:22.731725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:22.732595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:22.732595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:22.732595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:22.732595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:22.732595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:22.732595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:22.732595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:22.732595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:22.732595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:22.732595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:22.732595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:22.732595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:22.732595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:22.732595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:22.732595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:22.732595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:22.732595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:22.733425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:22.733425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:22.733425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:22.733425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:22.733425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 239/463] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:22.733693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a891a-7794-4fed-bfa2-a52f94e077cc None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:22.752822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 225/464] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:22.753721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cc3d9-ca91-47ff-bcd5-20ea3a61696d None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:22.754068 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cc3d9-ca91-47ff-bcd5-20ea3a61696d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:22.754412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cc3d9-ca91-47ff-bcd5-20ea3a61696d None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:22.755781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 226/465] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:22.756300 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cc3d9-ca91-47ff-bcd5-20ea3a61696d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:22.781258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:22.860615 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a891a-7794-4fed-bfa2-a52f94e077cc None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:22.878735 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cc3d9-ca91-47ff-bcd5-20ea3a61696d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:22.903694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1bade-916f-4b81-b4e7-482b8326e8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:22.904790 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1bade-916f-4b81-b4e7-482b8326e8ee None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:22.905069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1bade-916f-4b81-b4e7-482b8326e8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:22.905620 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1bade-916f-4b81-b4e7-482b8326e8ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:22.920049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1bade-916f-4b81-b4e7-482b8326e8ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:22.920853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10f04f-0590-44f4-9c22-a24322f29ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:22.921426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10f04f-0590-44f4-9c22-a24322f29ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:22.921623 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1bade-916f-4b81-b4e7-482b8326e8ee None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:22.921700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10f04f-0590-44f4-9c22-a24322f29ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:22.921990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10f04f-0590-44f4-9c22-a24322f29ec5 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:22.922759 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10f04f-0590-44f4-9c22-a24322f29ec5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:22.923070 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10f04f-0590-44f4-9c22-a24322f29ec5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:22.925333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10f04f-0590-44f4-9c22-a24322f29ec5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:22.927950 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:22.939530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-6ff1bade-916f-4b81-b4e7-482b8326e8ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:22.939530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:22.939530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:22.939530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:22.939530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:22.939530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:22.939530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:22.939530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:22.939530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:22.939530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:22.939530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:22.939530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:22.939530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:22.939530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:22.939530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:22.940552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:22.940552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:22.940552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:22.940552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:22.940552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:22.940552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:22.940552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:22.940552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:22.940552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:22.940552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:22.940552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:22.940552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:22.940552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:22.940552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:22.940552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:22.940552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:22.940552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:22.941399 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:22.941399 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:22.941399 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:22.941399 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:22.941606 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 227/466] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:22.941606 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10f04f-0590-44f4-9c22-a24322f29ec5 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:22.945447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 240/467] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:22.946738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121ef9e-9779-4a59-9863-4d587935df8d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:22.947060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121ef9e-9779-4a59-9863-4d587935df8d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:22.947336 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121ef9e-9779-4a59-9863-4d587935df8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:22.965589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f121ef9e-9779-4a59-9863-4d587935df8d None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:22.966831 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f121ef9e-9779-4a59-9863-4d587935df8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:23.042576 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 241/468] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:23.043756 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f121ef9e-9779-4a59-9863-4d587935df8d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:23.065003 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:23.081174 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10f04f-0590-44f4-9c22-a24322f29ec5 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:23.120914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83648b45-a9e4-493c-8987-ba3b9a53dc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:23.121668 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83648b45-a9e4-493c-8987-ba3b9a53dc78 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:23.122054 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83648b45-a9e4-493c-8987-ba3b9a53dc78 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:23.122392 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83648b45-a9e4-493c-8987-ba3b9a53dc78 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:23.123277 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83648b45-a9e4-493c-8987-ba3b9a53dc78 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:23.125061 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83648b45-a9e4-493c-8987-ba3b9a53dc78 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:23.149912 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 228/469] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Aug 1 15:20:22 2022] POST /identity/v3/users => generated 246 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:20:23.150936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83648b45-a9e4-493c-8987-ba3b9a53dc78 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:23.187027 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f121ef9e-9779-4a59-9863-4d587935df8d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:23.236166 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2aa16-6c40-4e55-8387-f6b04b3d1dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:23.237241 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2aa16-6c40-4e55-8387-f6b04b3d1dbc None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:23.237745 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2aa16-6c40-4e55-8387-f6b04b3d1dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:23.238266 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2aa16-6c40-4e55-8387-f6b04b3d1dbc None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:23.239076 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b2aa16-6c40-4e55-8387-f6b04b3d1dbc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:23.240757 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b2aa16-6c40-4e55-8387-f6b04b3d1dbc None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:23.243346 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:23.271062 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 242/470] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Aug 1 15:20:23 2022] POST /identity/v3/users => generated 248 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:20:23.272092 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2aa16-6c40-4e55-8387-f6b04b3d1dbc None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:23.359582 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83648b45-a9e4-493c-8987-ba3b9a53dc78 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:23.372688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:23.421052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2aa16-6c40-4e55-8387-f6b04b3d1dbc None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:23.423974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999626a-09d0-4177-a836-d4aed634579f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:23.426152 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999626a-09d0-4177-a836-d4aed634579f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:23.426475 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999626a-09d0-4177-a836-d4aed634579f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:23.426754 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999626a-09d0-4177-a836-d4aed634579f None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:23.427501 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999626a-09d0-4177-a836-d4aed634579f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:23.427805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999626a-09d0-4177-a836-d4aed634579f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:23.429266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999626a-09d0-4177-a836-d4aed634579f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:23.446504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 229/471] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:23.453087 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999626a-09d0-4177-a836-d4aed634579f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:23.488119 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637c0ed-f148-4476-92cc-1db925d3f797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:23.489558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637c0ed-f148-4476-92cc-1db925d3f797 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:23.491902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637c0ed-f148-4476-92cc-1db925d3f797 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:23.492808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637c0ed-f148-4476-92cc-1db925d3f797 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:23.510906 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d637c0ed-f148-4476-92cc-1db925d3f797 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:23.513062 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d637c0ed-f148-4476-92cc-1db925d3f797 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:23.539067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-d637c0ed-f148-4476-92cc-1db925d3f797 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:23.539067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:23.539067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:23.539067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:23.539067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:23.539067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:23.539067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:23.539067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:23.539067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:23.539067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:23.539067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:23.539067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:23.539067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:23.539067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:23.539067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:23.540215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:23.540215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:23.540215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:23.540215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:23.540215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:23.540215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:23.540215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:23.540215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:23.540215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:23.540215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:23.540215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:23.540215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:23.540215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:23.540215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:23.540215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:23.540215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:23.540215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:23.542033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:23.542033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:23.542033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:23.542033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:23.544307 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 243/472] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:23.546931 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9be27-bcbd-4940-97a2-1768e7ec1b6c None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:23.547542 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9be27-bcbd-4940-97a2-1768e7ec1b6c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:23.547885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9be27-bcbd-4940-97a2-1768e7ec1b6c None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:23.549866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 244/473] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:23.556838 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595dd6f-96d8-4c1e-8a28-a3c6f07bb404 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:23.557133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595dd6f-96d8-4c1e-8a28-a3c6f07bb404 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:23.557430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595dd6f-96d8-4c1e-8a28-a3c6f07bb404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:23.586449 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c595dd6f-96d8-4c1e-8a28-a3c6f07bb404 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:23.588565 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c595dd6f-96d8-4c1e-8a28-a3c6f07bb404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:23.605551 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999626a-09d0-4177-a836-d4aed634579f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:23.623404 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:23.649393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a72976-5a60-4c79-8d34-0a73ca3c5720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:23.650650 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a72976-5a60-4c79-8d34-0a73ca3c5720 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:23.650945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a72976-5a60-4c79-8d34-0a73ca3c5720 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:23.651223 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a72976-5a60-4c79-8d34-0a73ca3c5720 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:23.651912 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a72976-5a60-4c79-8d34-0a73ca3c5720 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:23.653498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a72976-5a60-4c79-8d34-0a73ca3c5720 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:23.684625 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 230/474] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Aug 1 15:20:23 2022] POST /identity/v3/users => generated 248 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:20:23.686095 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a72976-5a60-4c79-8d34-0a73ca3c5720 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:23.701949 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 245/475] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:23.755482 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595dd6f-96d8-4c1e-8a28-a3c6f07bb404 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:23.810360 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a72976-5a60-4c79-8d34-0a73ca3c5720 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:23.859201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316b6df-756f-4ffa-b5f0-084f9d4dd8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:23.859854 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316b6df-756f-4ffa-b5f0-084f9d4dd8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:23.861492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316b6df-756f-4ffa-b5f0-084f9d4dd8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:23.861792 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316b6df-756f-4ffa-b5f0-084f9d4dd8d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:23.862538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316b6df-756f-4ffa-b5f0-084f9d4dd8d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:23.862848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316b6df-756f-4ffa-b5f0-084f9d4dd8d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:23.864288 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316b6df-756f-4ffa-b5f0-084f9d4dd8d0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:23.882303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:23.894539 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 231/476] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:23.899653 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316b6df-756f-4ffa-b5f0-084f9d4dd8d0 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:23.956041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:23.997965 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595dd6f-96d8-4c1e-8a28-a3c6f07bb404 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:24.041347 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b0d3c-2916-4251-a13f-4a68f03aa7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:24.042215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b0d3c-2916-4251-a13f-4a68f03aa7fe None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:24.042600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b0d3c-2916-4251-a13f-4a68f03aa7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:24.042879 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b0d3c-2916-4251-a13f-4a68f03aa7fe None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:24.043612 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b0d3c-2916-4251-a13f-4a68f03aa7fe None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:24.044997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b0d3c-2916-4251-a13f-4a68f03aa7fe None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:24.062751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316b6df-756f-4ffa-b5f0-084f9d4dd8d0 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:24.067269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.sql.core [None req-cd1b0d3c-2916-4251-a13f-4a68f03aa7fe None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 15:20:24.067269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 15:20:24.067269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [parameters: {'id': '95f8f6c9d18843799ab10fdf3d216f4a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 15:20:24.067269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80348) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 15:20:24.086506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-cd1b0d3c-2916-4251-a13f-4a68f03aa7fe None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 15:20:24.086506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:24.086506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 15:20:24.086506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 15:20:24.086506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 15:20:24.086506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 15:20:24.086506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 15:20:24.086506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 15:20:24.086506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 15:20:24.086506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application conn.query(q) Aug 01 15:20:24.086506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 15:20:24.086506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 15:20:24.086506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 15:20:24.086506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application result.read() Aug 01 15:20:24.086506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 15:20:24.087688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 15:20:24.087688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 15:20:24.087688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 15:20:24.087688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 15:20:24.087688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 15:20:24.087688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 15:20:24.087688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 15:20:24.087688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 15:20:24.087688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:24.087688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 15:20:24.087688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:24.087688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:24.087688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 15:20:24.087688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 15:20:24.087688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 01 15:20:24.087688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return ref.to_dict() Aug 01 15:20:24.087688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 15:20:24.087688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application next(self.gen) Aug 01 15:20:24.088755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 15:20:24.088755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application yield resource Aug 01 15:20:24.088755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 15:20:24.088755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application next(self.gen) Aug 01 15:20:24.088755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 15:20:24.088755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 15:20:24.088755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 15:20:24.088755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 15:20:24.088755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 15:20:24.088755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise self.value Aug 01 15:20:24.088755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 15:20:24.088755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 15:20:24.088755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 15:20:24.088755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application session.commit() Aug 01 15:20:24.088755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 15:20:24.088755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 15:20:24.088755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 15:20:24.089618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 15:20:24.089618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 15:20:24.089618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self.session.flush() Aug 01 15:20:24.089618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 15:20:24.089618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 15:20:24.089618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 15:20:24.089618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 15:20:24.089618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 15:20:24.089618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application compat.raise_( Aug 01 15:20:24.089618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 15:20:24.089618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception Aug 01 15:20:24.089618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 15:20:24.089618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 15:20:24.089618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 15:20:24.089618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 15:20:24.089618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 15:20:24.089618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 15:20:24.090889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 15:20:24.090889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 15:20:24.090889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 15:20:24.090889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 15:20:24.090889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 15:20:24.090889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 15:20:24.090889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 15:20:24.090889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 15:20:24.090889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 15:20:24.090889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 15:20:24.090889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 15:20:24.090889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 15:20:24.090889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 15:20:24.090889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 15:20:24.090889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 15:20:24.090889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception Aug 01 15:20:24.091823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 15:20:24.091823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 15:20:24.091823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 15:20:24.091823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 15:20:24.091823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 15:20:24.091823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 15:20:24.091823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 15:20:24.091823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application conn.query(q) Aug 01 15:20:24.091823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 15:20:24.091823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 15:20:24.091823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 15:20:24.091823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application result.read() Aug 01 15:20:24.091823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 15:20:24.091823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 15:20:24.091823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 15:20:24.091823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 15:20:24.092667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 15:20:24.092667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 15:20:24.092667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 15:20:24.092667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 15:20:24.092667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 15:20:24.092667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 15:20:24.092667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [parameters: {'id': '95f8f6c9d18843799ab10fdf3d216f4a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 15:20:24.092667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 15:20:24.092667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:24.092667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:20:24.092667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:24.092667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:24.092667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:24.092667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:24.092667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:24.092667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:24.093469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:24.093469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:24.093469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:24.093469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:24.093469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:24.093469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:24.093469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 01 15:20:24.093469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 01 15:20:24.093469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:24.093469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:24.093469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 01 15:20:24.093469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 01 15:20:24.093469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 15:20:24.093469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 15:20:24.093469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 15:20:24.093469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:24.096171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 246/477] 10.222.0.83 () {60 vars in 1131 bytes} [Mon Aug 1 15:20:23 2022] POST /identity/v3/roles => generated 144 bytes in 341 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:20:24.105033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b0d3c-2916-4251-a13f-4a68f03aa7fe None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:24.123237 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebfa0f-3a5e-42d2-9bef-eb48bed18845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:24.123938 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebfa0f-3a5e-42d2-9bef-eb48bed18845 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:24.124270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebfa0f-3a5e-42d2-9bef-eb48bed18845 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:24.124552 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebfa0f-3a5e-42d2-9bef-eb48bed18845 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:24.125455 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebfa0f-3a5e-42d2-9bef-eb48bed18845 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:24.127234 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebfa0f-3a5e-42d2-9bef-eb48bed18845 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:24.162090 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 232/478] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Aug 1 15:20:23 2022] POST /identity/v3/users => generated 249 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:20:24.241182 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b0d3c-2916-4251-a13f-4a68f03aa7fe None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:24.258913 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:24.287269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a1f2c-1b22-42a0-9622-0f61d247736c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:24.288189 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a1f2c-1b22-42a0-9622-0f61d247736c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:24.288653 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a1f2c-1b22-42a0-9622-0f61d247736c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:24.289074 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a1f2c-1b22-42a0-9622-0f61d247736c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:24.303759 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a1f2c-1b22-42a0-9622-0f61d247736c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:24.309990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-760a1f2c-1b22-42a0-9622-0f61d247736c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:20:24.309990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:24.309990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:24.309990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:24.309990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:24.309990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:24.309990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:24.309990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:24.309990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:24.309990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:24.309990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:24.309990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:24.309990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:24.309990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:24.309990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:24.309990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:24.311187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:24.311187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:24.311187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:24.311187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:24.311187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:24.311187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:24.311187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:24.311187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:24.311187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:24.311187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:24.311187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:24.311187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:24.311187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:24.311187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:24.311187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:24.311187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:24.312173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:24.312173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:24.312173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:24.312173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:24.312173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:24.312173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:24.312173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:24.312173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:24.312173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:24.312173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:24.312173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:20:24.312173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:24.314557 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 247/479] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:20:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:24.323667 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebfa0f-3a5e-42d2-9bef-eb48bed18845 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:24.440743 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:24.458237 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebfa0f-3a5e-42d2-9bef-eb48bed18845 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:24.501614 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43062895-78d2-40bc-b670-5d3b6aa41811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:24.502698 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43062895-78d2-40bc-b670-5d3b6aa41811 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:24.503253 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43062895-78d2-40bc-b670-5d3b6aa41811 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:24.503791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43062895-78d2-40bc-b670-5d3b6aa41811 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:24.505109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43062895-78d2-40bc-b670-5d3b6aa41811 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:24.505579 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43062895-78d2-40bc-b670-5d3b6aa41811 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:24.508996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43062895-78d2-40bc-b670-5d3b6aa41811 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:24.522970 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 233/480] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:25.788775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabde86-1325-4a8b-95a2-7468d713709b None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:25.789319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabde86-1325-4a8b-95a2-7468d713709b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:25.789319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabde86-1325-4a8b-95a2-7468d713709b None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:25.791044 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 248/481] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:25.796358 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775031ec-0be8-4bc9-99e6-325f089a3e40 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:25.796767 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775031ec-0be8-4bc9-99e6-325f089a3e40 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:25.796909 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775031ec-0be8-4bc9-99e6-325f089a3e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:25.817443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-775031ec-0be8-4bc9-99e6-325f089a3e40 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:25.818663 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-775031ec-0be8-4bc9-99e6-325f089a3e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:25.839567 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457a5b7-30d7-4f3f-bf69-271ca8bfaac6 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:25.839842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457a5b7-30d7-4f3f-bf69-271ca8bfaac6 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:25.840103 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457a5b7-30d7-4f3f-bf69-271ca8bfaac6 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:25.841433 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 249/482] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:20:25.847660 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ea892-7fef-4cf7-b1aa-69e9c6597d6c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:25.848108 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ea892-7fef-4cf7-b1aa-69e9c6597d6c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:25.848541 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ea892-7fef-4cf7-b1aa-69e9c6597d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:25.887955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e55ea892-7fef-4cf7-b1aa-69e9c6597d6c None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:25.889798 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e55ea892-7fef-4cf7-b1aa-69e9c6597d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:25.938972 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 234/483] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:20:25.980826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775031ec-0be8-4bc9-99e6-325f089a3e40 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:26.038093 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 250/484] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:26.070785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ea892-7fef-4cf7-b1aa-69e9c6597d6c None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:26.181748 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:26.226848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775031ec-0be8-4bc9-99e6-325f089a3e40 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:26.243205 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ea892-7fef-4cf7-b1aa-69e9c6597d6c None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:26.246197 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:26.302498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08150278-951e-47aa-9466-3fd47451ffe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:26.303296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08150278-951e-47aa-9466-3fd47451ffe2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:26.303641 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08150278-951e-47aa-9466-3fd47451ffe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:26.303984 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08150278-951e-47aa-9466-3fd47451ffe2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:26.316592 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f3854-8518-473d-8f6e-ce2860b62c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:26.317297 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f3854-8518-473d-8f6e-ce2860b62c88 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:26.317645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f3854-8518-473d-8f6e-ce2860b62c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:26.318005 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f3854-8518-473d-8f6e-ce2860b62c88 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:26.335915 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08150278-951e-47aa-9466-3fd47451ffe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:26.344753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-08150278-951e-47aa-9466-3fd47451ffe2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:26.344753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:26.344753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:26.344753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:26.344753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:26.344753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:26.344753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:26.344753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:26.344753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:26.344753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:26.344753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:26.344753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:26.344753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:26.344753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:26.344753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:26.344753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:26.345752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:26.345752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:26.345752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:26.345752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:26.345752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:26.345752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:26.345752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:26.345752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:26.345752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:26.345752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:26.345752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:26.345752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:26.345752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:26.345752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:26.345752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:26.345752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:26.346949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:26.346949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:26.346949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:26.346949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:26.346949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:26.346949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:26.346949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:26.346949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:26.346949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:26.346949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:26.346949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:26.346949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:26.351161 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f3854-8518-473d-8f6e-ce2860b62c88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:26.357778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 235/485] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:20:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:26.357923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-332f3854-8518-473d-8f6e-ce2860b62c88 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:26.357923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:26.357923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:26.357923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:26.357923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:26.357923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:26.357923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:26.357923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:26.357923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:26.357923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:26.357923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:26.357923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:26.357923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:26.357923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:26.357923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:26.357923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:26.358796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:26.358796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:26.358796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:26.358796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:26.358796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:26.358796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:26.358796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:26.358796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:26.358796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:26.358796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:26.358796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:26.358796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:26.358796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:26.358796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:26.358796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:26.358796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:26.359617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:26.359617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:26.359617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:26.359617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:26.359617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:26.359617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:26.359617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:26.359617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:26.359617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:26.359617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:26.359617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:26.359617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:26.359617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 251/486] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:20:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:20:26.365105 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08150278-951e-47aa-9466-3fd47451ffe2 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:26.371355 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f3854-8518-473d-8f6e-ce2860b62c88 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:26.516121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f3854-8518-473d-8f6e-ce2860b62c88 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:26.519653 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08150278-951e-47aa-9466-3fd47451ffe2 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:26.551561 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b737f1b-b7b3-4112-b300-c35dbb01b057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:26.552023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b737f1b-b7b3-4112-b300-c35dbb01b057 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:26.552305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b737f1b-b7b3-4112-b300-c35dbb01b057 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:26.552577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b737f1b-b7b3-4112-b300-c35dbb01b057 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:26.553305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b737f1b-b7b3-4112-b300-c35dbb01b057 None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:26.553594 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b737f1b-b7b3-4112-b300-c35dbb01b057 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:26.561912 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b737f1b-b7b3-4112-b300-c35dbb01b057 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:26.565122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 252/487] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:20:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:26.565869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4edfa3-fef6-4518-9cae-4d853d6060c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:26.566806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4edfa3-fef6-4518-9cae-4d853d6060c4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:26.567326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4edfa3-fef6-4518-9cae-4d853d6060c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:26.567798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4edfa3-fef6-4518-9cae-4d853d6060c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:26.568723 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4edfa3-fef6-4518-9cae-4d853d6060c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:26.569304 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4edfa3-fef6-4518-9cae-4d853d6060c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:26.570858 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b737f1b-b7b3-4112-b300-c35dbb01b057 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:26.572359 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4edfa3-fef6-4518-9cae-4d853d6060c4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:26.583545 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:26.592524 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 236/488] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:20:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:26.598046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4edfa3-fef6-4518-9cae-4d853d6060c4 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:26.602381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:26.756411 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b737f1b-b7b3-4112-b300-c35dbb01b057 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:26.817942 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4edfa3-fef6-4518-9cae-4d853d6060c4 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:26.818443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a83211-4e42-48da-959b-0eb111c6c83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:26.818792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a83211-4e42-48da-959b-0eb111c6c83d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:26.819030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a83211-4e42-48da-959b-0eb111c6c83d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:26.819348 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a83211-4e42-48da-959b-0eb111c6c83d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:26.832192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a83211-4e42-48da-959b-0eb111c6c83d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:26.833782 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a83211-4e42-48da-959b-0eb111c6c83d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:26.850009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-68a83211-4e42-48da-959b-0eb111c6c83d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:26.850009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:26.850009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:26.850009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:26.850009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:26.850009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:26.850009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:26.850009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:26.850009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:26.850009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:26.850009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:26.850009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:26.850009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:26.850009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:26.850009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:26.850932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:26.850932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:26.850932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:26.850932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:26.850932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:26.850932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:26.850932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:26.850932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:26.850932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:26.850932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:26.850932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:26.850932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:26.850932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:26.850932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:26.850932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:26.850932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:26.850932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:26.851768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:26.851768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:26.851768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:26.851768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:26.851987 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 253/489] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:20:26.853614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108827fc-e846-4710-8d49-23ff564958fa None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:26.853911 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108827fc-e846-4710-8d49-23ff564958fa None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:26.854204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108827fc-e846-4710-8d49-23ff564958fa None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:26.859568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 254/490] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:26.860433 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108827fc-e846-4710-8d49-23ff564958fa None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:26.874013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d3b47-f33d-4c0a-a71f-dd43c9d47a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:26.874766 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d3b47-f33d-4c0a-a71f-dd43c9d47a5d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:26.875384 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d3b47-f33d-4c0a-a71f-dd43c9d47a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:26.875664 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d3b47-f33d-4c0a-a71f-dd43c9d47a5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:26.890722 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d3b47-f33d-4c0a-a71f-dd43c9d47a5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:26.893066 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d3b47-f33d-4c0a-a71f-dd43c9d47a5d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:26.914351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-612d3b47-f33d-4c0a-a71f-dd43c9d47a5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:26.914351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:26.914351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:26.914351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:26.914351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:26.914351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:26.914351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:26.914351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:26.914351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:26.914351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:26.914351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:26.914351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:26.914351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:26.914351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:26.914351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:26.915271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:26.915271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:26.915271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:26.915271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:26.915271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:26.915271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:26.915271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:26.915271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:26.915271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:26.915271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:26.915271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:26.915271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:26.915271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:26.915271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:26.915271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:26.915271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:26.915271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:26.916682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:26.916682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:26.916682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:26.916682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:26.916901 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 237/491] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:20:26.918839 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc07d7-be76-4888-a85d-6cfa9ec01fd2 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:26.919183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc07d7-be76-4888-a85d-6cfa9ec01fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:26.919466 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc07d7-be76-4888-a85d-6cfa9ec01fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:26.942637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6ecc07d7-be76-4888-a85d-6cfa9ec01fd2 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:26.943947 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6ecc07d7-be76-4888-a85d-6cfa9ec01fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:26.957351 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:26.990605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108827fc-e846-4710-8d49-23ff564958fa None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:27.014187 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:27.033047 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2aabb7-b89f-4af4-941d-7f0d6bfcf1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:27.033725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2aabb7-b89f-4af4-941d-7f0d6bfcf1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:27.034086 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2aabb7-b89f-4af4-941d-7f0d6bfcf1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:27.034438 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2aabb7-b89f-4af4-941d-7f0d6bfcf1d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:27.035173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2aabb7-b89f-4af4-941d-7f0d6bfcf1d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:27.035477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2aabb7-b89f-4af4-941d-7f0d6bfcf1d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:27.036968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2aabb7-b89f-4af4-941d-7f0d6bfcf1d7 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:27.046145 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 238/492] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:27.046145 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc07d7-be76-4888-a85d-6cfa9ec01fd2 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:27.051625 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 255/493] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:27.053169 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8006f0-31b1-44d8-ada3-466194ce33db None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:27.053594 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8006f0-31b1-44d8-ada3-466194ce33db None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:27.053999 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8006f0-31b1-44d8-ada3-466194ce33db None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:27.055688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 256/494] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:27.059581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8006f0-31b1-44d8-ada3-466194ce33db None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:27.163575 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc07d7-be76-4888-a85d-6cfa9ec01fd2 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:27.195282 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8006f0-31b1-44d8-ada3-466194ce33db None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:27.208613 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344b5f1-269c-48aa-898c-24982f2989db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:27.209269 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344b5f1-269c-48aa-898c-24982f2989db None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:27.209782 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344b5f1-269c-48aa-898c-24982f2989db None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:27.210090 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344b5f1-269c-48aa-898c-24982f2989db None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:27.210800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344b5f1-269c-48aa-898c-24982f2989db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:27.211104 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344b5f1-269c-48aa-898c-24982f2989db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:27.213023 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344b5f1-269c-48aa-898c-24982f2989db None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:27.227569 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 239/495] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:27.229440 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb029ec-6ac7-4485-8bcc-750ec784d388 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:27.229761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb029ec-6ac7-4485-8bcc-750ec784d388 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:27.230066 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb029ec-6ac7-4485-8bcc-750ec784d388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:27.245018 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea3a07-9774-4de7-a14a-790524a50d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:27.245708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea3a07-9774-4de7-a14a-790524a50d4c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:27.246025 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea3a07-9774-4de7-a14a-790524a50d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:27.246299 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea3a07-9774-4de7-a14a-790524a50d4c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:27.249738 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-1fb029ec-6ac7-4485-8bcc-750ec784d388 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:27.249881 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:27.251180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1fb029ec-6ac7-4485-8bcc-750ec784d388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:27.257386 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea3a07-9774-4de7-a14a-790524a50d4c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:27.259285 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea3a07-9774-4de7-a14a-790524a50d4c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:27.276278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-b6ea3a07-9774-4de7-a14a-790524a50d4c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:20:27.276278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:27.276278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:27.276278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:27.276278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:27.276278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:27.276278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:27.276278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:27.276278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:27.276278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:27.276278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:27.276278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:27.276278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:20:27.276278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:20:27.276278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:20:27.276278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:20:27.277381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:27.277381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:27.277381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:20:27.277381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:27.277381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:20:27.277381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:27.277381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:27.277381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:27.277381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:27.277381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:27.277381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:27.277381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:27.277381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:27.277381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:27.277381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:27.277381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:27.277381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:27.277381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:27.278516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:27.278516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:27.278516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:20:27.278516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:20:27.278516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:20:27.278516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:20:27.278516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:20:27.278516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:20:27.278516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:20:27.278516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:27.284477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 257/496] 10.222.0.83 () {56 vars in 1093 bytes} [Mon Aug 1 15:20:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:27.284477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea3a07-9774-4de7-a14a-790524a50d4c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:27.296160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:27.341795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 240/497] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:27.343235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb029ec-6ac7-4485-8bcc-750ec784d388 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:27.401339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea3a07-9774-4de7-a14a-790524a50d4c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:27.452795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e071ed-09bd-4d61-a364-c777ce1be6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:27.453577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e071ed-09bd-4d61-a364-c777ce1be6cd None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:27.454061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e071ed-09bd-4d61-a364-c777ce1be6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:27.454504 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e071ed-09bd-4d61-a364-c777ce1be6cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:27.472145 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e071ed-09bd-4d61-a364-c777ce1be6cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:27.482133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-05e071ed-09bd-4d61-a364-c777ce1be6cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:27.482133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:27.482133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:27.482133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:27.482133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:27.482133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:27.482133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:27.482133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:27.482133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:27.482133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:27.482133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:27.482133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:27.482133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:27.482133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:27.482133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:27.482133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:27.483508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:27.483508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:27.483508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:27.483508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:27.483508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:27.483508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:27.483508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:27.483508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:27.483508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:27.483508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:27.483508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:27.483508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:27.483508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:27.483508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:27.483508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:27.483508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:27.484694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:27.484694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:27.484694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:27.484694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:27.484694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:27.484694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:27.484694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:27.484694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:27.484694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:27.484694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:27.484694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:27.484694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:27.484694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 258/498] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:20:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:27.485621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e071ed-09bd-4d61-a364-c777ce1be6cd None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:27.509835 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb029ec-6ac7-4485-8bcc-750ec784d388 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:27.562889 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a989cfa5-d8c3-458c-a8fc-0452ec4ee549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:27.563195 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:27.563709 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989cfa5-d8c3-458c-a8fc-0452ec4ee549 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:27.563994 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989cfa5-d8c3-458c-a8fc-0452ec4ee549 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:27.564372 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989cfa5-d8c3-458c-a8fc-0452ec4ee549 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:27.573232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a989cfa5-d8c3-458c-a8fc-0452ec4ee549 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:27.574975 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a989cfa5-d8c3-458c-a8fc-0452ec4ee549 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:27.588870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-a989cfa5-d8c3-458c-a8fc-0452ec4ee549 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 15:20:27.588870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:27.588870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:27.588870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:27.588870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:27.588870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:27.588870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:27.588870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:27.588870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:27.588870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:27.588870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:27.588870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:27.588870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:20:27.588870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:20:27.588870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:20:27.588870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:20:27.590017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:27.590017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:27.590017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:20:27.590017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:27.590017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:20:27.590017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:27.590017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:27.590017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:27.590017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:27.590017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:27.590017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:27.590017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:27.590017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:27.590017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:27.590017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:27.590017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:27.590017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:27.590017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:27.595817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:27.595817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:27.595817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:20:27.595817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:20:27.595817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:20:27.595817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:20:27.595817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:20:27.595817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:20:27.595817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 15:20:27.595817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:27.595817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 241/499] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:27.595817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b80bae-fc08-4661-a407-2832b7939177 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:27.595817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b80bae-fc08-4661-a407-2832b7939177 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:27.596533 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b80bae-fc08-4661-a407-2832b7939177 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:27.596533 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 242/500] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:27.596533 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b80bae-fc08-4661-a407-2832b7939177 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:27.616524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e071ed-09bd-4d61-a364-c777ce1be6cd None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:27.626563 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:27.654777 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b26cd-807e-474e-a512-71b9be3e7c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:27.655401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b26cd-807e-474e-a512-71b9be3e7c3f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:27.655681 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b26cd-807e-474e-a512-71b9be3e7c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:27.655955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b26cd-807e-474e-a512-71b9be3e7c3f None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:27.656763 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b26cd-807e-474e-a512-71b9be3e7c3f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:27.657069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b26cd-807e-474e-a512-71b9be3e7c3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:27.658566 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b26cd-807e-474e-a512-71b9be3e7c3f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:27.695518 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 259/501] 10.222.0.83 () {56 vars in 1133 bytes} [Mon Aug 1 15:20:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:27.696356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b26cd-807e-474e-a512-71b9be3e7c3f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:27.718248 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b80bae-fc08-4661-a407-2832b7939177 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:27.754823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51ccf3-aa56-440a-a373-bff366f8d8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:27.755468 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51ccf3-aa56-440a-a373-bff366f8d8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:27.755750 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51ccf3-aa56-440a-a373-bff366f8d8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:27.756025 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51ccf3-aa56-440a-a373-bff366f8d8a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:27.769267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51ccf3-aa56-440a-a373-bff366f8d8a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:27.778969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-1b51ccf3-aa56-440a-a373-bff366f8d8a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:27.778969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:27.778969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:27.778969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:27.778969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:27.778969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:27.778969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:27.778969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:27.778969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:27.778969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:27.778969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:27.778969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:27.778969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:27.778969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:27.778969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:27.778969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:27.779892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:27.779892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:27.779892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:27.779892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:27.779892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:27.779892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:27.779892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:27.779892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:27.779892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:27.779892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:27.779892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:27.779892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:27.779892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:27.779892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:27.779892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:27.779892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:27.780677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:27.780677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:27.780677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:27.780677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:27.780677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:27.780677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:27.780677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:27.780677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:27.780677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:27.780677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:27.780677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:27.780677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:27.781428 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 243/502] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:20:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:27.781997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51ccf3-aa56-440a-a373-bff366f8d8a5 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:27.805687 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b26cd-807e-474e-a512-71b9be3e7c3f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:27.820843 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:27.845304 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00288eaa-92ec-4bd1-9590-0b259b473bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:27.845962 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00288eaa-92ec-4bd1-9590-0b259b473bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:27.846250 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00288eaa-92ec-4bd1-9590-0b259b473bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:27.846522 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00288eaa-92ec-4bd1-9590-0b259b473bc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:27.847243 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00288eaa-92ec-4bd1-9590-0b259b473bc5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:27.847545 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00288eaa-92ec-4bd1-9590-0b259b473bc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:27.848939 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00288eaa-92ec-4bd1-9590-0b259b473bc5 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:27.857743 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 260/503] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:20:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:27.859214 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746d9e0-cc76-4d73-a742-1e28a0ce301f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:27.859591 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746d9e0-cc76-4d73-a742-1e28a0ce301f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:27.859915 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746d9e0-cc76-4d73-a742-1e28a0ce301f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:27.881480 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-3746d9e0-cc76-4d73-a742-1e28a0ce301f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:27.882726 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3746d9e0-cc76-4d73-a742-1e28a0ce301f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:27.894697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:27.905485 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51ccf3-aa56-440a-a373-bff366f8d8a5 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:27.941310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb6337-85cc-4a9e-babc-3fce3e73ad48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:27.942029 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb6337-85cc-4a9e-babc-3fce3e73ad48 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:27.942391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb6337-85cc-4a9e-babc-3fce3e73ad48 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:27.942709 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb6337-85cc-4a9e-babc-3fce3e73ad48 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:27.943769 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb6337-85cc-4a9e-babc-3fce3e73ad48 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:27.944151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb6337-85cc-4a9e-babc-3fce3e73ad48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:27.946328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb6337-85cc-4a9e-babc-3fce3e73ad48 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:27.952432 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 261/504] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:27.953273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746d9e0-cc76-4d73-a742-1e28a0ce301f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:27.984871 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 244/505] 10.222.0.83 () {56 vars in 1137 bytes} [Mon Aug 1 15:20:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:27.986047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb6337-85cc-4a9e-babc-3fce3e73ad48 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:28.066508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746d9e0-cc76-4d73-a742-1e28a0ce301f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:28.075495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:28.096585 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb6337-85cc-4a9e-babc-3fce3e73ad48 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:28.102782 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855249c-9dae-40ea-b00c-e050d493cf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:28.103435 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855249c-9dae-40ea-b00c-e050d493cf49 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:28.103717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855249c-9dae-40ea-b00c-e050d493cf49 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:28.103990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855249c-9dae-40ea-b00c-e050d493cf49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:28.132463 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66d47a-c012-49f8-a239-c31762531920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:28.133279 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66d47a-c012-49f8-a239-c31762531920 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:28.133630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66d47a-c012-49f8-a239-c31762531920 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:28.134056 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66d47a-c012-49f8-a239-c31762531920 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:28.135404 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66d47a-c012-49f8-a239-c31762531920 None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:28.135792 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66d47a-c012-49f8-a239-c31762531920 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:28.137193 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66d47a-c012-49f8-a239-c31762531920 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:28.145452 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 245/506] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:20:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:28.146296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66d47a-c012-49f8-a239-c31762531920 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:28.159051 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:28.174557 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855249c-9dae-40ea-b00c-e050d493cf49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:28.176015 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855249c-9dae-40ea-b00c-e050d493cf49 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:28.252853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66d47a-c012-49f8-a239-c31762531920 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:28.282134 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 262/507] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:28.282983 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855249c-9dae-40ea-b00c-e050d493cf49 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:28.291507 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:28.292616 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65114f09-89a6-459c-8a07-0ff76b5e2ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:28.293315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65114f09-89a6-459c-8a07-0ff76b5e2ead None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:28.293632 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65114f09-89a6-459c-8a07-0ff76b5e2ead None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:28.294109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65114f09-89a6-459c-8a07-0ff76b5e2ead None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:28.305372 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65114f09-89a6-459c-8a07-0ff76b5e2ead None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:28.307428 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65114f09-89a6-459c-8a07-0ff76b5e2ead None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:28.323122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-65114f09-89a6-459c-8a07-0ff76b5e2ead None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:28.323122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:28.323122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:28.323122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:28.323122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:28.323122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:28.323122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:28.323122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:28.323122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:28.323122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:28.323122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:28.323122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:28.323122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:28.323122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:28.323122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:28.324434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:28.324434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:28.324434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:28.324434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:28.324434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:28.324434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:28.324434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:28.324434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:28.324434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:28.324434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:28.324434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:28.324434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:28.324434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:28.324434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:28.324434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:28.324434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:28.324434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:28.325810 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:28.325810 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:28.325810 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:28.325810 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:28.325810 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 246/508] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:28.325810 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65114f09-89a6-459c-8a07-0ff76b5e2ead None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:28.390458 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855249c-9dae-40ea-b00c-e050d493cf49 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:28.394412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:28.429306 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addac1ef-c6eb-4bbf-a6e4-ff9f3e460b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:28.429971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addac1ef-c6eb-4bbf-a6e4-ff9f3e460b70 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:28.430250 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addac1ef-c6eb-4bbf-a6e4-ff9f3e460b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:28.430519 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addac1ef-c6eb-4bbf-a6e4-ff9f3e460b70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:28.440125 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65114f09-89a6-459c-8a07-0ff76b5e2ead None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:28.441878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addac1ef-c6eb-4bbf-a6e4-ff9f3e460b70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:28.443674 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addac1ef-c6eb-4bbf-a6e4-ff9f3e460b70 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:28.460864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-addac1ef-c6eb-4bbf-a6e4-ff9f3e460b70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:28.460864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:28.460864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:28.460864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:28.460864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:28.460864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:28.460864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:28.460864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:28.460864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:28.460864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:28.460864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:28.460864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:28.460864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:28.460864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:28.460864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:28.461971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:28.461971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:28.461971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:28.461971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:28.461971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:28.461971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:28.461971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:28.461971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:28.461971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:28.461971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:28.461971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:28.461971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:28.461971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:28.461971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:28.461971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:28.461971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:28.461971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:28.463023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:28.463023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:28.463023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:28.463023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:28.463023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 263/509] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:28.463350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addac1ef-c6eb-4bbf-a6e4-ff9f3e460b70 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:28.477658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ec982-9c4c-40f8-ad4b-a777d41aa2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:28.478879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ec982-9c4c-40f8-ad4b-a777d41aa2fa None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:28.479159 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ec982-9c4c-40f8-ad4b-a777d41aa2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:28.479434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ec982-9c4c-40f8-ad4b-a777d41aa2fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:28.527957 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:28.552129 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ec982-9c4c-40f8-ad4b-a777d41aa2fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:28.553861 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ec982-9c4c-40f8-ad4b-a777d41aa2fa None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:28.575685 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addac1ef-c6eb-4bbf-a6e4-ff9f3e460b70 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:28.611877 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adabc532-fdae-42cf-99a7-0b84b6c363f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:28.612523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabc532-fdae-42cf-99a7-0b84b6c363f1 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:28.612802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabc532-fdae-42cf-99a7-0b84b6c363f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:28.613073 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabc532-fdae-42cf-99a7-0b84b6c363f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:28.615995 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:28.627146 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adabc532-fdae-42cf-99a7-0b84b6c363f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:28.629009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adabc532-fdae-42cf-99a7-0b84b6c363f1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:28.645697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-adabc532-fdae-42cf-99a7-0b84b6c363f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:28.645697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:28.645697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:28.645697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:28.645697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:28.645697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:28.645697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:28.645697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:28.645697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:28.645697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:28.645697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:28.645697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:28.645697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:28.645697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:28.645697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:28.646857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:28.646857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:28.646857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:28.646857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:28.646857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:28.646857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:28.646857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:28.646857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:28.646857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:28.646857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:28.646857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:28.646857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:28.646857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:28.646857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:28.646857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:28.646857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:28.646857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:28.647745 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:28.647745 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:28.647745 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:28.647745 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:28.647745 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 264/510] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:28.648162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adabc532-fdae-42cf-99a7-0b84b6c363f1 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:28.662586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 247/511] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:28.663437 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ec982-9c4c-40f8-ad4b-a777d41aa2fa None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:28.751327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adabc532-fdae-42cf-99a7-0b84b6c363f1 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:28.762007 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:28.773273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ec982-9c4c-40f8-ad4b-a777d41aa2fa None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:28.798165 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3144ef9-4d17-4174-a66b-3d76e5dcf1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:28.798818 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3144ef9-4d17-4174-a66b-3d76e5dcf1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:28.799211 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3144ef9-4d17-4174-a66b-3d76e5dcf1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:28.799497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3144ef9-4d17-4174-a66b-3d76e5dcf1e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:28.808923 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b064e51-7fc9-426c-a428-7a3dffe01518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:28.809546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b064e51-7fc9-426c-a428-7a3dffe01518 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:28.809826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b064e51-7fc9-426c-a428-7a3dffe01518 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:28.810141 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b064e51-7fc9-426c-a428-7a3dffe01518 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:28.810837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b064e51-7fc9-426c-a428-7a3dffe01518 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:28.811138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b064e51-7fc9-426c-a428-7a3dffe01518 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:28.811469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3144ef9-4d17-4174-a66b-3d76e5dcf1e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:28.812535 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b064e51-7fc9-426c-a428-7a3dffe01518 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:28.812850 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3144ef9-4d17-4174-a66b-3d76e5dcf1e2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:28.825532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 248/512] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:28.826406 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b064e51-7fc9-426c-a428-7a3dffe01518 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:28.831586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-c3144ef9-4d17-4174-a66b-3d76e5dcf1e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:28.831586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:28.831586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:28.831586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:28.831586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:28.831586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:28.831586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:28.831586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:28.831586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:28.831586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:28.831586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:28.831586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:28.831586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:28.831586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:28.831586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:28.833546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:28.833546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:28.833546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:28.833546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:28.833546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:28.833546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:28.833546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:28.833546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:28.833546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:28.833546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:28.833546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:28.833546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:28.833546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:28.833546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:28.833546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:28.833546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:28.833546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:28.835006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:28.835006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:28.835006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:28.835006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:28.835006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 265/513] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:28.835006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3144ef9-4d17-4174-a66b-3d76e5dcf1e2 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:28.854671 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:28.938477 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b064e51-7fc9-426c-a428-7a3dffe01518 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:28.956497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3144ef9-4d17-4174-a66b-3d76e5dcf1e2 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:28.974146 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea39dd9-0063-486d-bb19-a0f4aa1b643f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:28.974777 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea39dd9-0063-486d-bb19-a0f4aa1b643f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:28.975057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea39dd9-0063-486d-bb19-a0f4aa1b643f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:28.975334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea39dd9-0063-486d-bb19-a0f4aa1b643f None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:28.976031 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea39dd9-0063-486d-bb19-a0f4aa1b643f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:28.976333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea39dd9-0063-486d-bb19-a0f4aa1b643f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:28.977717 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea39dd9-0063-486d-bb19-a0f4aa1b643f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:28.990139 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 249/514] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:28.991055 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea39dd9-0063-486d-bb19-a0f4aa1b643f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:28.997294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1d52b-b98b-45d9-ad15-9e32a4a8e80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:28.998059 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1d52b-b98b-45d9-ad15-9e32a4a8e80c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:28.998392 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1d52b-b98b-45d9-ad15-9e32a4a8e80c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:28.998739 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1d52b-b98b-45d9-ad15-9e32a4a8e80c None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:28.999533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de1d52b-b98b-45d9-ad15-9e32a4a8e80c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:28.999845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de1d52b-b98b-45d9-ad15-9e32a4a8e80c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:29.001346 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de1d52b-b98b-45d9-ad15-9e32a4a8e80c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:29.016242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 266/515] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:29.017046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1d52b-b98b-45d9-ad15-9e32a4a8e80c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:29.042107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:29.083264 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:29.107702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea39dd9-0063-486d-bb19-a0f4aa1b643f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:29.128369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1d52b-b98b-45d9-ad15-9e32a4a8e80c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:29.147480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76008da-7c3a-4f20-9778-8a9219122831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:29.148346 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76008da-7c3a-4f20-9778-8a9219122831 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:29.148716 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76008da-7c3a-4f20-9778-8a9219122831 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:29.149092 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76008da-7c3a-4f20-9778-8a9219122831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:29.162039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76008da-7c3a-4f20-9778-8a9219122831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:29.164774 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76008da-7c3a-4f20-9778-8a9219122831 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:29.165490 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87c810-dd0d-4ee1-a4b3-e7a46ded3e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:29.166187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87c810-dd0d-4ee1-a4b3-e7a46ded3e79 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:29.166456 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87c810-dd0d-4ee1-a4b3-e7a46ded3e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:29.166720 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87c810-dd0d-4ee1-a4b3-e7a46ded3e79 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:29.178577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87c810-dd0d-4ee1-a4b3-e7a46ded3e79 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:29.180054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87c810-dd0d-4ee1-a4b3-e7a46ded3e79 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:29.187398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-a76008da-7c3a-4f20-9778-8a9219122831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:29.187398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:29.187398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:29.187398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:29.187398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:29.187398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:29.187398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:29.187398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:29.187398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:29.187398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:29.187398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:29.187398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:29.187398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:29.187398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:29.187398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:29.188450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:29.188450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:29.188450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:29.188450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:29.188450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:29.188450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:29.188450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:29.188450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:29.188450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:29.188450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:29.188450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:29.188450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:29.188450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:29.188450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:29.188450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:29.188450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:29.188450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:29.189483 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:29.189483 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:29.189483 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:29.189483 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:29.189483 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 250/516] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:29.190858 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76008da-7c3a-4f20-9778-8a9219122831 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:29.193327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-0e87c810-dd0d-4ee1-a4b3-e7a46ded3e79 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:20:29.193327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:29.193327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:29.193327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:29.193327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:29.193327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:29.193327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:29.193327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:29.193327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:29.193327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:29.193327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:29.193327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:29.193327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:20:29.193327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:20:29.193327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:20:29.193327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:20:29.194257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:29.194257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:29.194257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:20:29.194257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:29.194257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:20:29.194257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:29.194257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:29.194257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:29.194257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:29.194257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:29.194257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:29.194257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:29.194257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:29.194257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:29.194257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:29.194257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:29.194257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:29.194257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:29.195412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:29.195412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:29.195412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:20:29.195412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:20:29.195412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:20:29.195412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:20:29.195412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:20:29.195412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:20:29.195412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:20:29.195412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:29.195412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 267/517] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:29.195984 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87c810-dd0d-4ee1-a4b3-e7a46ded3e79 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:29.291438 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:29.308112 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76008da-7c3a-4f20-9778-8a9219122831 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:29.310698 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87c810-dd0d-4ee1-a4b3-e7a46ded3e79 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:29.344874 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c93db0-1878-40a2-841b-3a82f7f4f146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:29.345489 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c93db0-1878-40a2-841b-3a82f7f4f146 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:29.345764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c93db0-1878-40a2-841b-3a82f7f4f146 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:29.346064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c93db0-1878-40a2-841b-3a82f7f4f146 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:29.346741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c93db0-1878-40a2-841b-3a82f7f4f146 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:29.348193 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c93db0-1878-40a2-841b-3a82f7f4f146 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:29.348340 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37b168-5adf-4d97-9eb2-6c1b69200e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:29.349081 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37b168-5adf-4d97-9eb2-6c1b69200e3f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:29.349472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37b168-5adf-4d97-9eb2-6c1b69200e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:29.349757 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37b168-5adf-4d97-9eb2-6c1b69200e3f None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:29.350702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37b168-5adf-4d97-9eb2-6c1b69200e3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:29.351016 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37b168-5adf-4d97-9eb2-6c1b69200e3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:29.352855 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37b168-5adf-4d97-9eb2-6c1b69200e3f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:29.354640 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:29.366706 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 251/518] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:29.367532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37b168-5adf-4d97-9eb2-6c1b69200e3f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:29.374899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 268/519] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Aug 1 15:20:29 2022] POST /identity/v3/users => generated 247 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:20:29.375817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c93db0-1878-40a2-841b-3a82f7f4f146 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:29.512644 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37b168-5adf-4d97-9eb2-6c1b69200e3f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:29.516036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c93db0-1878-40a2-841b-3a82f7f4f146 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:29.566620 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d965d-8143-422e-8b49-626605160c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:29.567188 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d965d-8143-422e-8b49-626605160c59 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:29.567468 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d965d-8143-422e-8b49-626605160c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:29.568073 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d965d-8143-422e-8b49-626605160c59 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:29.577188 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3daff-4d56-4945-905b-cb00e1703b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:29.578398 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3daff-4d56-4945-905b-cb00e1703b4a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:29.578961 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3daff-4d56-4945-905b-cb00e1703b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:29.579322 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d965d-8143-422e-8b49-626605160c59 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:29.579634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3daff-4d56-4945-905b-cb00e1703b4a None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:29.580789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d965d-8143-422e-8b49-626605160c59 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:29.580901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3daff-4d56-4945-905b-cb00e1703b4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:29.581487 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3daff-4d56-4945-905b-cb00e1703b4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:29.583037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3daff-4d56-4945-905b-cb00e1703b4a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:29.591104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:29.593720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-377d965d-8143-422e-8b49-626605160c59 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 15:20:29.593720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:29.593720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:29.593720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:29.593720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:29.593720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:29.593720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:29.593720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:29.593720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:29.593720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:29.593720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:29.593720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:29.593720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:20:29.593720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:20:29.593720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:20:29.593720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:20:29.594822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:29.594822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:29.594822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:20:29.594822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:29.594822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:20:29.594822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:29.594822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:29.594822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:29.594822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:29.594822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:29.594822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:29.594822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:29.594822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:29.594822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:29.594822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:29.594822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:29.594822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:29.594822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:29.595868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:29.595868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:29.595868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:20:29.595868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:20:29.595868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:20:29.595868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:20:29.595868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:20:29.595868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:20:29.595868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 15:20:29.595868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:29.595868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 252/520] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:20:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:29.596544 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d965d-8143-422e-8b49-626605160c59 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:29.601001 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 269/521] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:29.603286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3daff-4d56-4945-905b-cb00e1703b4a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:29.658620 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:29.708990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d965d-8143-422e-8b49-626605160c59 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:29.724691 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3daff-4d56-4945-905b-cb00e1703b4a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:29.746426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6529f2-4361-4068-ae35-c1ddbac5ae3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:29.747407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6529f2-4361-4068-ae35-c1ddbac5ae3b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:29.747767 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6529f2-4361-4068-ae35-c1ddbac5ae3b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:29.748119 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6529f2-4361-4068-ae35-c1ddbac5ae3b None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:29.749023 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6529f2-4361-4068-ae35-c1ddbac5ae3b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:29.749355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6529f2-4361-4068-ae35-c1ddbac5ae3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:29.750824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6529f2-4361-4068-ae35-c1ddbac5ae3b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:29.764419 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c3d24-d1d8-4cec-bb07-9a4bcd76934c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:29.765199 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c3d24-d1d8-4cec-bb07-9a4bcd76934c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:29.765538 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c3d24-d1d8-4cec-bb07-9a4bcd76934c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:29.765872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c3d24-d1d8-4cec-bb07-9a4bcd76934c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:29.777726 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 253/522] 10.222.0.83 () {56 vars in 1137 bytes} [Mon Aug 1 15:20:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:29.777930 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c3d24-d1d8-4cec-bb07-9a4bcd76934c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:29.778023 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6529f2-4361-4068-ae35-c1ddbac5ae3b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:29.778131 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c3d24-d1d8-4cec-bb07-9a4bcd76934c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:29.791888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-812c3d24-d1d8-4cec-bb07-9a4bcd76934c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:29.791888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:29.791888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:29.791888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:29.791888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:29.791888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:29.791888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:29.791888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:29.791888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:29.791888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:29.791888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:29.791888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:29.791888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:20:29.791888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:20:29.791888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:20:29.793252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:20:29.793252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:29.793252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:29.793252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:29.793252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:29.793252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:29.793252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:29.793252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:29.793252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:29.793252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:29.793252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:29.793252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:29.793252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:29.793252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:29.793252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:29.793252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:29.793252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:29.794387 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:20:29.794387 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:20:29.794387 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:20:29.794387 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:20:29.794387 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:20:29.794387 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:20:29.794387 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:29.794387 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:29.799889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 270/523] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:29.799889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c3d24-d1d8-4cec-bb07-9a4bcd76934c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:29.879931 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:29.913464 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6529f2-4361-4068-ae35-c1ddbac5ae3b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:29.930443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:29.943953 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c3d24-d1d8-4cec-bb07-9a4bcd76934c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:29.958194 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69956344-7b0d-49af-8cb2-8c6c1bfea508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:29.959010 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69956344-7b0d-49af-8cb2-8c6c1bfea508 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:29.959293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69956344-7b0d-49af-8cb2-8c6c1bfea508 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:29.959570 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69956344-7b0d-49af-8cb2-8c6c1bfea508 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:29.960363 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69956344-7b0d-49af-8cb2-8c6c1bfea508 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:29.960666 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69956344-7b0d-49af-8cb2-8c6c1bfea508 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:29.962163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69956344-7b0d-49af-8cb2-8c6c1bfea508 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:29.982270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 254/524] 10.222.0.83 () {56 vars in 1139 bytes} [Mon Aug 1 15:20:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:29.982270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69956344-7b0d-49af-8cb2-8c6c1bfea508 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:29.991963 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d42f9-0863-4c54-a34d-26c2799a7038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:29.992719 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d42f9-0863-4c54-a34d-26c2799a7038 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:29.993068 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d42f9-0863-4c54-a34d-26c2799a7038 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:29.993519 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d42f9-0863-4c54-a34d-26c2799a7038 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:30.003985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d42f9-0863-4c54-a34d-26c2799a7038 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:30.005652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d42f9-0863-4c54-a34d-26c2799a7038 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:30.018796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-8d1d42f9-0863-4c54-a34d-26c2799a7038 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:30.018796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:30.018796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:30.018796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:30.018796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:30.018796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:30.018796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:30.018796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:30.018796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:30.018796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:30.018796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:30.018796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:30.018796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:20:30.018796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:20:30.018796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:20:30.020032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:20:30.020032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:30.020032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:30.020032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:30.020032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:30.020032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:30.020032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:30.020032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:30.020032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:30.020032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:30.020032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:30.020032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:30.020032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:30.020032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:30.020032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:30.020032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:30.020032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:30.021072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:20:30.021072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:20:30.021072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:20:30.021072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:20:30.021072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:20:30.021072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:20:30.021072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:30.021072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:30.021072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 271/525] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:30.021072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d42f9-0863-4c54-a34d-26c2799a7038 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:30.118582 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69956344-7b0d-49af-8cb2-8c6c1bfea508 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:30.143265 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d42f9-0863-4c54-a34d-26c2799a7038 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:30.161647 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89be0ec-fbfa-4b22-854e-dddeece05c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:30.162731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89be0ec-fbfa-4b22-854e-dddeece05c37 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:30.163171 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89be0ec-fbfa-4b22-854e-dddeece05c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:30.163690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89be0ec-fbfa-4b22-854e-dddeece05c37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:30.164319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:30.189574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889c68b-ab66-44dd-b343-0954657bb7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:30.190553 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889c68b-ab66-44dd-b343-0954657bb7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:30.190837 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889c68b-ab66-44dd-b343-0954657bb7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:30.191111 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889c68b-ab66-44dd-b343-0954657bb7e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:30.191858 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889c68b-ab66-44dd-b343-0954657bb7e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:30.192164 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889c68b-ab66-44dd-b343-0954657bb7e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:30.194790 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889c68b-ab66-44dd-b343-0954657bb7e2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:30.210911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:30.219647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 272/526] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:20:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:30.220417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889c68b-ab66-44dd-b343-0954657bb7e2 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:30.252150 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89be0ec-fbfa-4b22-854e-dddeece05c37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:30.254180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89be0ec-fbfa-4b22-854e-dddeece05c37 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:30.343951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889c68b-ab66-44dd-b343-0954657bb7e2 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:30.369369 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 255/527] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:30.370276 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89be0ec-fbfa-4b22-854e-dddeece05c37 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:30.395973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17720d6f-1112-42b6-a317-6e9feecb5ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:30.396620 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17720d6f-1112-42b6-a317-6e9feecb5ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:30.396967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17720d6f-1112-42b6-a317-6e9feecb5ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:30.397246 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17720d6f-1112-42b6-a317-6e9feecb5ce6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:30.448044 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:30.454869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:30.475098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17720d6f-1112-42b6-a317-6e9feecb5ce6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:30.476550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17720d6f-1112-42b6-a317-6e9feecb5ce6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:30.495178 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89be0ec-fbfa-4b22-854e-dddeece05c37 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:30.535609 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52549f2b-94a8-48f1-8fae-fbb446ee5d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:30.536244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52549f2b-94a8-48f1-8fae-fbb446ee5d73 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:30.536634 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52549f2b-94a8-48f1-8fae-fbb446ee5d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:30.536784 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52549f2b-94a8-48f1-8fae-fbb446ee5d73 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:30.537548 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52549f2b-94a8-48f1-8fae-fbb446ee5d73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:30.537853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52549f2b-94a8-48f1-8fae-fbb446ee5d73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:30.539355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52549f2b-94a8-48f1-8fae-fbb446ee5d73 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:30.558447 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 256/528] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:20:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:30.559082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52549f2b-94a8-48f1-8fae-fbb446ee5d73 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:30.601937 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 273/529] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:30.601937 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17720d6f-1112-42b6-a317-6e9feecb5ce6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:30.687236 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52549f2b-94a8-48f1-8fae-fbb446ee5d73 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:30.693489 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:30.722668 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17720d6f-1112-42b6-a317-6e9feecb5ce6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:30.724020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:30.731844 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109480c0-c6ee-4001-9074-81372a76712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:30.732543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109480c0-c6ee-4001-9074-81372a76712a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:30.733120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109480c0-c6ee-4001-9074-81372a76712a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:30.733120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109480c0-c6ee-4001-9074-81372a76712a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:30.734073 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109480c0-c6ee-4001-9074-81372a76712a None admin] RBAC: Adding query filter params (role.id=091d39715077464daa4cf890db55e6b9, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=25dc41ffc41c47fdbeec06110d04610c) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:30.734438 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109480c0-c6ee-4001-9074-81372a76712a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:30.736097 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109480c0-c6ee-4001-9074-81372a76712a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:30.753921 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 257/530] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:20:30 2022] GET /identity/v3/role_assignments?user.id=25dc41ffc41c47fdbeec06110d04610c&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=091d39715077464daa4cf890db55e6b9 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:30.753921 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109480c0-c6ee-4001-9074-81372a76712a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:30.776493 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e17cb-a731-4140-a30e-05eb42b6fc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:30.777312 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e17cb-a731-4140-a30e-05eb42b6fc9e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:30.777971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e17cb-a731-4140-a30e-05eb42b6fc9e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:30.778347 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e17cb-a731-4140-a30e-05eb42b6fc9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:30.793669 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e17cb-a731-4140-a30e-05eb42b6fc9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:30.795314 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e17cb-a731-4140-a30e-05eb42b6fc9e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:30.818550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-f86e17cb-a731-4140-a30e-05eb42b6fc9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:30.818550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:30.818550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:30.818550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:30.818550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:30.818550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:30.818550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:30.818550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:30.818550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:30.818550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:30.818550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:30.818550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:30.818550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:30.818550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:30.818550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:30.819820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:30.819820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:30.819820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:30.819820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:30.819820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:30.819820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:30.819820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:30.819820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:30.819820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:30.819820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:30.819820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:30.819820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:30.819820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:30.819820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:30.819820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:30.819820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:30.819820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:30.821193 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:30.821193 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:30.821193 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:30.821193 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:30.822793 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 274/531] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:30.823878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e17cb-a731-4140-a30e-05eb42b6fc9e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:30.912017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109480c0-c6ee-4001-9074-81372a76712a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:30.958420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d0856-1aa9-4c6f-81d3-4f7aa4d687b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:30.959041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d0856-1aa9-4c6f-81d3-4f7aa4d687b3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:30.959120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d0856-1aa9-4c6f-81d3-4f7aa4d687b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:30.959368 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d0856-1aa9-4c6f-81d3-4f7aa4d687b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:30.960249 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d0856-1aa9-4c6f-81d3-4f7aa4d687b3 None admin] RBAC: Adding query filter params (role.id=091d39715077464daa4cf890db55e6b9, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=69d216a568b546589e854c2b4476efe6) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:30.960640 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d0856-1aa9-4c6f-81d3-4f7aa4d687b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:30.962134 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d0856-1aa9-4c6f-81d3-4f7aa4d687b3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:30.969009 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 258/532] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:20:30 2022] GET /identity/v3/role_assignments?user.id=69d216a568b546589e854c2b4476efe6&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=091d39715077464daa4cf890db55e6b9 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:30.969919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d0856-1aa9-4c6f-81d3-4f7aa4d687b3 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:30.981262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e17cb-a731-4140-a30e-05eb42b6fc9e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:31.017929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:31.027647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540e95a-9d8a-49e5-b9b8-db06263dee91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:31.028921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540e95a-9d8a-49e5-b9b8-db06263dee91 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:31.029528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540e95a-9d8a-49e5-b9b8-db06263dee91 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:31.030253 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540e95a-9d8a-49e5-b9b8-db06263dee91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:31.045255 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540e95a-9d8a-49e5-b9b8-db06263dee91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:31.047731 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540e95a-9d8a-49e5-b9b8-db06263dee91 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:31.049930 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:31.080584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-3540e95a-9d8a-49e5-b9b8-db06263dee91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:31.080584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:31.080584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:31.080584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:31.080584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:31.080584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:31.080584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:31.080584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:31.080584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:31.080584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:31.080584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:31.080584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:31.080584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:31.080584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:31.080584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:31.081593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:31.081593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:31.081593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:31.081593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:31.081593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:31.081593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:31.081593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:31.081593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:31.081593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:31.081593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:31.081593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:31.081593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:31.081593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:31.081593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:31.081593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:31.081593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:31.081593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:31.082622 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:31.082622 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:31.082622 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:31.082622 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:31.085268 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 275/533] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:31.092094 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540e95a-9d8a-49e5-b9b8-db06263dee91 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:31.108640 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d0856-1aa9-4c6f-81d3-4f7aa4d687b3 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:31.202711 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc8baa-0df9-44cf-92f7-57ed782c63f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:31.203356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc8baa-0df9-44cf-92f7-57ed782c63f5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:31.203643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc8baa-0df9-44cf-92f7-57ed782c63f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:31.203918 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc8baa-0df9-44cf-92f7-57ed782c63f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:31.204633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc8baa-0df9-44cf-92f7-57ed782c63f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:31.204940 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc8baa-0df9-44cf-92f7-57ed782c63f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:31.206394 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc8baa-0df9-44cf-92f7-57ed782c63f5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:31.221129 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540e95a-9d8a-49e5-b9b8-db06263dee91 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:31.236143 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 259/534] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:31.243183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc8baa-0df9-44cf-92f7-57ed782c63f5 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:31.261550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e96d2-b8ee-4445-ab0d-0e2f875eb6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:31.262669 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e96d2-b8ee-4445-ab0d-0e2f875eb6aa None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:31.263273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e96d2-b8ee-4445-ab0d-0e2f875eb6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:31.263746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e96d2-b8ee-4445-ab0d-0e2f875eb6aa None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:31.264659 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e96d2-b8ee-4445-ab0d-0e2f875eb6aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:31.265192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e96d2-b8ee-4445-ab0d-0e2f875eb6aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:31.266852 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e96d2-b8ee-4445-ab0d-0e2f875eb6aa None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:31.281102 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 276/535] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:31.286855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e96d2-b8ee-4445-ab0d-0e2f875eb6aa None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:31.336191 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:31.399407 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e96d2-b8ee-4445-ab0d-0e2f875eb6aa None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:31.438765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39727092-c42c-4d53-a486-afbb62eec896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:31.439810 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39727092-c42c-4d53-a486-afbb62eec896 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:31.440294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39727092-c42c-4d53-a486-afbb62eec896 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:31.440784 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39727092-c42c-4d53-a486-afbb62eec896 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:31.449418 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39727092-c42c-4d53-a486-afbb62eec896 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:31.451234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39727092-c42c-4d53-a486-afbb62eec896 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:31.464703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-39727092-c42c-4d53-a486-afbb62eec896 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:31.464703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:31.464703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:31.464703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:31.464703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:31.464703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:31.464703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:31.464703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:31.464703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:31.464703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:31.464703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:31.464703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:31.464703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:20:31.464703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:20:31.464703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:20:31.465621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:20:31.465621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:31.465621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:31.465621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:31.465621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:31.465621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:31.465621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:31.465621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:31.465621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:31.465621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:31.465621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:31.465621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:31.465621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:31.465621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:31.465621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:31.465621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:31.465621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:31.466667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:20:31.466667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:20:31.466667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:20:31.466667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:20:31.466667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:20:31.466667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:20:31.466667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:31.466667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:31.468846 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 277/536] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:20:31.474926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39727092-c42c-4d53-a486-afbb62eec896 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:31.511737 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc8baa-0df9-44cf-92f7-57ed782c63f5 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:31.547934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:31.585233 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc46064-642c-4002-a700-ebf68b50931b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:31.586571 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc46064-642c-4002-a700-ebf68b50931b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:31.587116 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc46064-642c-4002-a700-ebf68b50931b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:31.588136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc46064-642c-4002-a700-ebf68b50931b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:31.607282 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc46064-642c-4002-a700-ebf68b50931b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:31.609131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc46064-642c-4002-a700-ebf68b50931b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:31.627517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-bfc46064-642c-4002-a700-ebf68b50931b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:31.627517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:31.627517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:31.627517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:31.627517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:31.627517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:31.627517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:31.627517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:31.627517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:31.627517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:31.627517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:31.627517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:31.627517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:20:31.627517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:20:31.627517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:20:31.628443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:20:31.628443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:31.628443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:31.628443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:31.628443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:31.628443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:31.628443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:31.628443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:31.628443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:31.628443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:31.628443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:31.628443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:31.628443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:31.628443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:31.628443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:31.628443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:31.628443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:31.629372 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:20:31.629372 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:20:31.629372 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:20:31.629372 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:20:31.629372 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:20:31.629372 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:20:31.629372 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:31.629372 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:31.629781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 260/537] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:31.631025 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb72a7e-9625-4d14-a284-5594107e829d None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:31.631427 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb72a7e-9625-4d14-a284-5594107e829d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:31.631757 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb72a7e-9625-4d14-a284-5594107e829d None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:31.638427 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 261/538] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:31.639982 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb72a7e-9625-4d14-a284-5594107e829d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:31.646692 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39727092-c42c-4d53-a486-afbb62eec896 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:31.649052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:31.694695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726e965-555f-44e1-b3a5-f4a20774884d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:31.695681 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726e965-555f-44e1-b3a5-f4a20774884d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:31.695965 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726e965-555f-44e1-b3a5-f4a20774884d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:31.696326 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726e965-555f-44e1-b3a5-f4a20774884d None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:31.697166 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726e965-555f-44e1-b3a5-f4a20774884d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:31.697585 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726e965-555f-44e1-b3a5-f4a20774884d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:31.699397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726e965-555f-44e1-b3a5-f4a20774884d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:31.714732 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 278/539] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:20:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:31.716177 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f4e00-1234-425b-ac7d-9274268c76d4 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:31.716470 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f4e00-1234-425b-ac7d-9274268c76d4 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:31.716809 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f4e00-1234-425b-ac7d-9274268c76d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:31.743083 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-833f4e00-1234-425b-ac7d-9274268c76d4 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:31.744396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-833f4e00-1234-425b-ac7d-9274268c76d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:31.770840 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb72a7e-9625-4d14-a284-5594107e829d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:31.811776 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f6b0d-6a0c-42d0-aff7-5672224408eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:31.812864 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f6b0d-6a0c-42d0-aff7-5672224408eb None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:31.813352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f6b0d-6a0c-42d0-aff7-5672224408eb None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:31.813852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f6b0d-6a0c-42d0-aff7-5672224408eb None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:31.814905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f6b0d-6a0c-42d0-aff7-5672224408eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:31.815539 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f6b0d-6a0c-42d0-aff7-5672224408eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:31.817298 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f6b0d-6a0c-42d0-aff7-5672224408eb None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:31.824305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 279/540] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:31.825394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f4e00-1234-425b-ac7d-9274268c76d4 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:31.831912 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 262/541] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:20:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:31.838608 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f6b0d-6a0c-42d0-aff7-5672224408eb None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:31.885045 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:31.934993 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:31.947410 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f4e00-1234-425b-ac7d-9274268c76d4 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:31.960326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f6b0d-6a0c-42d0-aff7-5672224408eb None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:31.993368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4060b3-f898-4c76-ab31-61f14d6de1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:31.995345 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4060b3-f898-4c76-ab31-61f14d6de1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:31.996149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4060b3-f898-4c76-ab31-61f14d6de1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:31.996673 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4060b3-f898-4c76-ab31-61f14d6de1c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:31.998882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4060b3-f898-4c76-ab31-61f14d6de1c0 None admin] RBAC: Adding query filter params (role.id=091d39715077464daa4cf890db55e6b9, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=76311c7173b442c0af2c745e3e70a07d) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:31.999828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4060b3-f898-4c76-ab31-61f14d6de1c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:32.002107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4060b3-f898-4c76-ab31-61f14d6de1c0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:32.017927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 280/542] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:20:31 2022] GET /identity/v3/role_assignments?user.id=76311c7173b442c0af2c745e3e70a07d&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=091d39715077464daa4cf890db55e6b9 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:32.017927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4060b3-f898-4c76-ab31-61f14d6de1c0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:32.018966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709d1e8-9a2c-4734-869d-e84c48f56d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:32.021141 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709d1e8-9a2c-4734-869d-e84c48f56d94 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:32.021293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709d1e8-9a2c-4734-869d-e84c48f56d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:32.024587 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709d1e8-9a2c-4734-869d-e84c48f56d94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:32.024587 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709d1e8-9a2c-4734-869d-e84c48f56d94 None admin] RBAC: Adding query filter params (role.id=091d39715077464daa4cf890db55e6b9, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=f54b570990794801a5e45918845569b8) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:32.024587 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709d1e8-9a2c-4734-869d-e84c48f56d94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:32.027623 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709d1e8-9a2c-4734-869d-e84c48f56d94 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:32.041699 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 263/543] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:20:31 2022] GET /identity/v3/role_assignments?user.id=f54b570990794801a5e45918845569b8&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=091d39715077464daa4cf890db55e6b9 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:32.220607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4060b3-f898-4c76-ab31-61f14d6de1c0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:32.270092 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c229b-e58a-4751-a27d-63f6c6396964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:32.270676 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c229b-e58a-4751-a27d-63f6c6396964 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:32.270952 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c229b-e58a-4751-a27d-63f6c6396964 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:32.271221 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c229b-e58a-4751-a27d-63f6c6396964 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:32.289299 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c229b-e58a-4751-a27d-63f6c6396964 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:32.295704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-739c229b-e58a-4751-a27d-63f6c6396964 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:32.295704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:32.295704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:32.295704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:32.295704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:32.295704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:32.295704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:32.295704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:32.295704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:32.295704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:32.295704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:32.295704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:32.295704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:32.295704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:32.295704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:32.295704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:32.296725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:32.296725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:32.296725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:32.296725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:32.296725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:32.296725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:32.296725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:32.296725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:32.296725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:32.296725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:32.296725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:32.296725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:32.296725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:32.296725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:32.296725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:32.296725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:32.297663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:32.297663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:32.297663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:32.297663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:32.297663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:32.297663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:32.297663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:32.297663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:32.297663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:32.297663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:32.297663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:32.297663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:32.297663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 281/544] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:20:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:32.302897 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709d1e8-9a2c-4734-869d-e84c48f56d94 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:32.413618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709d1e8-9a2c-4734-869d-e84c48f56d94 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:32.415416 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:32.454451 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd1f93-8ebe-4667-84b1-736a5bf224c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:32.455511 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd1f93-8ebe-4667-84b1-736a5bf224c4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:32.455944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd1f93-8ebe-4667-84b1-736a5bf224c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:32.456416 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd1f93-8ebe-4667-84b1-736a5bf224c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:32.457659 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd1f93-8ebe-4667-84b1-736a5bf224c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:32.458296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd1f93-8ebe-4667-84b1-736a5bf224c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:32.459752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd1f93-8ebe-4667-84b1-736a5bf224c4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:32.469854 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 264/545] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:20:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:32.475717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c229b-e58a-4751-a27d-63f6c6396964 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:32.504465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:32.617839 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c229b-e58a-4751-a27d-63f6c6396964 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:32.662670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b19213-28d0-48ea-9aad-3fe7ba27ac8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:32.663326 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b19213-28d0-48ea-9aad-3fe7ba27ac8f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:32.663612 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b19213-28d0-48ea-9aad-3fe7ba27ac8f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:32.663889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b19213-28d0-48ea-9aad-3fe7ba27ac8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:32.681390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b19213-28d0-48ea-9aad-3fe7ba27ac8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:32.682901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b19213-28d0-48ea-9aad-3fe7ba27ac8f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:32.701780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-08b19213-28d0-48ea-9aad-3fe7ba27ac8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:32.701780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:32.701780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:32.701780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:32.701780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:32.701780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:32.701780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:32.701780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:32.701780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:32.701780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:32.701780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:32.701780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:32.701780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:32.701780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:32.701780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:32.702893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:32.702893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:32.702893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:32.702893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:32.702893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:32.702893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:32.702893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:32.702893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:32.702893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:32.702893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:32.702893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:32.702893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:32.702893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:32.702893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:32.702893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:32.702893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:32.702893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:32.703804 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:32.703804 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:32.703804 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:32.703804 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:32.703804 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 282/546] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:32.708778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd1f93-8ebe-4667-84b1-736a5bf224c4 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:32.831947 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd1f93-8ebe-4667-84b1-736a5bf224c4 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:32.868298 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6528f-80b9-4c03-80e7-28c60d4d2e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:32.868897 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6528f-80b9-4c03-80e7-28c60d4d2e7d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:32.869187 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6528f-80b9-4c03-80e7-28c60d4d2e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:32.869468 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6528f-80b9-4c03-80e7-28c60d4d2e7d None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:32.870207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6528f-80b9-4c03-80e7-28c60d4d2e7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:32.870518 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6528f-80b9-4c03-80e7-28c60d4d2e7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:32.872015 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6528f-80b9-4c03-80e7-28c60d4d2e7d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:32.886841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 265/547] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:32.895343 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b19213-28d0-48ea-9aad-3fe7ba27ac8f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:33.002747 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:33.036537 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b19213-28d0-48ea-9aad-3fe7ba27ac8f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:33.076965 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafd6ba-cde7-434f-8290-e04dd842ccf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:33.077619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafd6ba-cde7-434f-8290-e04dd842ccf4 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:33.077918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafd6ba-cde7-434f-8290-e04dd842ccf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:33.078197 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafd6ba-cde7-434f-8290-e04dd842ccf4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:33.090454 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acafd6ba-cde7-434f-8290-e04dd842ccf4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:33.092113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acafd6ba-cde7-434f-8290-e04dd842ccf4 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:33.111078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-acafd6ba-cde7-434f-8290-e04dd842ccf4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:20:33.111078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:33.111078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:33.111078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:33.111078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:33.111078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:33.111078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:33.111078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:33.111078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:33.111078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:33.111078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:33.111078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:33.111078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:20:33.111078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:20:33.111078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:20:33.111078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:20:33.112036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:33.112036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:33.112036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:20:33.112036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:33.112036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:20:33.112036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:33.112036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:33.112036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:33.112036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:33.112036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:33.112036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:33.112036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:33.112036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:33.112036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:33.112036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:33.112036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:33.112036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:33.112036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:33.113457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:33.113457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:33.113457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:20:33.113457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:20:33.113457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:20:33.113457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:20:33.113457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:20:33.113457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:20:33.113457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:20:33.113457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:33.113457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 283/548] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:20:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:33.122046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6528f-80b9-4c03-80e7-28c60d4d2e7d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:33.178310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:33.308238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6528f-80b9-4c03-80e7-28c60d4d2e7d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:33.360994 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90396399-8b0f-457d-8c4a-ac0706a7d45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:33.362043 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90396399-8b0f-457d-8c4a-ac0706a7d45f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:33.362524 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90396399-8b0f-457d-8c4a-ac0706a7d45f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:33.363028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90396399-8b0f-457d-8c4a-ac0706a7d45f None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:33.364040 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90396399-8b0f-457d-8c4a-ac0706a7d45f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:33.364550 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90396399-8b0f-457d-8c4a-ac0706a7d45f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:33.366241 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90396399-8b0f-457d-8c4a-ac0706a7d45f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:33.393620 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 266/549] 10.222.0.83 () {56 vars in 1135 bytes} [Mon Aug 1 15:20:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:33.399762 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90396399-8b0f-457d-8c4a-ac0706a7d45f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:33.400145 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0892129-34fc-42af-b7c5-2ddb12575a38 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:33.400437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0892129-34fc-42af-b7c5-2ddb12575a38 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:33.400716 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0892129-34fc-42af-b7c5-2ddb12575a38 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:33.403477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 284/550] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:33.407960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654b2c0-d1d1-4d5c-a2e7-5f4af55d68b8 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:33.408242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654b2c0-d1d1-4d5c-a2e7-5f4af55d68b8 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:33.408576 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654b2c0-d1d1-4d5c-a2e7-5f4af55d68b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:33.433316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-3654b2c0-d1d1-4d5c-a2e7-5f4af55d68b8 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:33.437431 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3654b2c0-d1d1-4d5c-a2e7-5f4af55d68b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:33.546711 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:33.549083 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 285/551] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:33.567043 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90396399-8b0f-457d-8c4a-ac0706a7d45f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:33.571588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654b2c0-d1d1-4d5c-a2e7-5f4af55d68b8 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:33.618393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9fa2b-98c6-47bc-8839-c13ebabf39a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:33.619039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9fa2b-98c6-47bc-8839-c13ebabf39a4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:33.619319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9fa2b-98c6-47bc-8839-c13ebabf39a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:33.620069 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9fa2b-98c6-47bc-8839-c13ebabf39a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:33.642829 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:33.697729 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654b2c0-d1d1-4d5c-a2e7-5f4af55d68b8 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:33.722148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9fa2b-98c6-47bc-8839-c13ebabf39a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:33.723918 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9fa2b-98c6-47bc-8839-c13ebabf39a4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:33.742144 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa904bd-341e-4138-a18b-1f1bdabef58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:33.743463 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa904bd-341e-4138-a18b-1f1bdabef58f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:33.744168 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa904bd-341e-4138-a18b-1f1bdabef58f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:33.744588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa904bd-341e-4138-a18b-1f1bdabef58f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:33.762664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa904bd-341e-4138-a18b-1f1bdabef58f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:33.768549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-efa904bd-341e-4138-a18b-1f1bdabef58f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:33.768549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:33.768549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:33.768549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:33.768549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:33.768549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:33.768549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:33.768549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:33.768549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:33.768549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:33.768549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:33.768549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:33.768549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:33.768549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:33.768549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:33.768549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:33.769795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:33.769795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:33.769795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:33.769795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:33.769795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:33.769795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:33.769795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:33.769795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:33.769795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:33.769795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:33.769795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:33.769795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:33.769795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:33.769795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:33.769795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:33.769795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:33.770826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:33.770826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:33.770826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:33.770826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:33.770826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:33.770826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:33.770826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:33.770826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:33.770826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:33.770826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:33.770826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:33.770826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:33.775627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 286/552] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:20:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:20:33.777285 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a2eaa-9ba6-48d0-b98f-d2f1432b9d6d None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:33.777583 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a2eaa-9ba6-48d0-b98f-d2f1432b9d6d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:33.777953 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a2eaa-9ba6-48d0-b98f-d2f1432b9d6d None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:33.783408 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 287/553] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:33.784510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a2eaa-9ba6-48d0-b98f-d2f1432b9d6d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:33.832807 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:33.855412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 267/554] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 15:20:33.861554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54955c88-f487-4fb9-9566-2e07c07b5594 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:33.861934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54955c88-f487-4fb9-9566-2e07c07b5594 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:33.862330 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54955c88-f487-4fb9-9566-2e07c07b5594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:33.881711 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-54955c88-f487-4fb9-9566-2e07c07b5594 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:33.883292 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-54955c88-f487-4fb9-9566-2e07c07b5594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:33.911133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a2eaa-9ba6-48d0-b98f-d2f1432b9d6d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:33.930793 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:33.954240 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec3e8c-0cfa-4dc8-b03d-b99f2b3dc512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:33.955176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec3e8c-0cfa-4dc8-b03d-b99f2b3dc512 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:33.955648 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec3e8c-0cfa-4dc8-b03d-b99f2b3dc512 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:33.956088 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec3e8c-0cfa-4dc8-b03d-b99f2b3dc512 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:33.956969 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec3e8c-0cfa-4dc8-b03d-b99f2b3dc512 None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:33.957421 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec3e8c-0cfa-4dc8-b03d-b99f2b3dc512 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:33.959262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec3e8c-0cfa-4dc8-b03d-b99f2b3dc512 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:33.973911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 268/555] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:33.973911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54955c88-f487-4fb9-9566-2e07c07b5594 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:33.976677 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 288/556] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:20:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:33.982834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec3e8c-0cfa-4dc8-b03d-b99f2b3dc512 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:34.087127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54955c88-f487-4fb9-9566-2e07c07b5594 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:34.105995 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:34.127106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec3e8c-0cfa-4dc8-b03d-b99f2b3dc512 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:34.134727 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdcfde-22f3-4e0a-af96-ff77417fbc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:34.135504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdcfde-22f3-4e0a-af96-ff77417fbc7e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:34.135855 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdcfde-22f3-4e0a-af96-ff77417fbc7e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:34.136198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdcfde-22f3-4e0a-af96-ff77417fbc7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:34.153760 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdcfde-22f3-4e0a-af96-ff77417fbc7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:34.155665 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdcfde-22f3-4e0a-af96-ff77417fbc7e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:34.170409 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e60f4-d298-462f-ab79-43fa37613955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:34.171306 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e60f4-d298-462f-ab79-43fa37613955 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:34.171808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e60f4-d298-462f-ab79-43fa37613955 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:34.172234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e60f4-d298-462f-ab79-43fa37613955 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:34.176398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-b6bdcfde-22f3-4e0a-af96-ff77417fbc7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:34.176398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:34.176398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:34.176398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:34.176398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:34.176398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:34.176398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:34.176398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:34.176398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:34.176398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:34.176398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:34.176398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:34.176398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:34.176398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:34.176398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:34.177510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:34.177510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:34.177510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:34.177510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:34.177510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:34.177510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:34.177510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:34.177510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:34.177510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:34.177510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:34.177510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:34.177510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:34.177510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:34.177510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:34.177510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:34.177510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:34.177510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:34.178659 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:34.178659 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:34.178659 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:34.178659 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:34.185247 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 269/557] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:34.186498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdcfde-22f3-4e0a-af96-ff77417fbc7e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:34.189741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e60f4-d298-462f-ab79-43fa37613955 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:34.191867 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e60f4-d298-462f-ab79-43fa37613955 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:34.209695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-1b2e60f4-d298-462f-ab79-43fa37613955 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:34.209695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:34.209695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:34.209695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:34.209695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:34.209695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:34.209695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:34.209695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:34.209695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:34.209695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:34.209695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:34.209695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:34.209695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:34.209695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:34.209695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:34.211365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:34.211365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:34.211365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:34.211365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:34.211365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:34.211365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:34.211365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:34.211365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:34.211365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:34.211365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:34.211365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:34.211365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:34.211365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:34.211365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:34.211365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:34.211365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:34.211365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:34.213052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:34.213052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:34.213052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:34.213052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:34.219568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 289/558] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:34.220657 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e60f4-d298-462f-ab79-43fa37613955 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:34.257108 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:34.331880 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdcfde-22f3-4e0a-af96-ff77417fbc7e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:34.339730 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e60f4-d298-462f-ab79-43fa37613955 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:34.378484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ee1a8-219e-4860-8153-3185b9d12bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:34.379363 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ee1a8-219e-4860-8153-3185b9d12bbe None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:34.379795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ee1a8-219e-4860-8153-3185b9d12bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:34.380087 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4a863-7368-41bb-ab34-e18999bc6927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:34.380416 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ee1a8-219e-4860-8153-3185b9d12bbe None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:34.381182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4a863-7368-41bb-ab34-e18999bc6927 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:34.381442 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ee1a8-219e-4860-8153-3185b9d12bbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:34.381731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4a863-7368-41bb-ab34-e18999bc6927 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:34.382099 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ee1a8-219e-4860-8153-3185b9d12bbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:34.382211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4a863-7368-41bb-ab34-e18999bc6927 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:34.383711 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ee1a8-219e-4860-8153-3185b9d12bbe None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:34.396327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd4a863-7368-41bb-ab34-e18999bc6927 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:34.402278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-9dd4a863-7368-41bb-ab34-e18999bc6927 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:34.402278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:34.402278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:34.402278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:34.402278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:34.402278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:34.402278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:34.402278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:34.402278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:34.402278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:34.402278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:34.402278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:34.402278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:34.402278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:34.402278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:34.402278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:34.403244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:34.403244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:34.403244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:34.403244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:34.403244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:34.403244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:34.403244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:34.403244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:34.403244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:34.403244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:34.403244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:34.403244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:34.403244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:34.403244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:34.403244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:34.403244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:34.404177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:34.404177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:34.404177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:34.404177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:34.404177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:34.404177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:34.404177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:34.404177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:34.404177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:34.404177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:34.404177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:34.404177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:34.405499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 290/559] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:34.406280 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ee1a8-219e-4860-8153-3185b9d12bbe None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:34.407484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 270/560] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:20:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:34.408506 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4a863-7368-41bb-ab34-e18999bc6927 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:34.439025 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:34.519077 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:34.530393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ee1a8-219e-4860-8153-3185b9d12bbe None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:34.531393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4a863-7368-41bb-ab34-e18999bc6927 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:34.569699 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a51919-8e9f-4324-af1d-39f24910e058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:34.570192 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41094a-aa38-406b-b7b6-33f608441895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:34.570624 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a51919-8e9f-4324-af1d-39f24910e058 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:34.570905 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a51919-8e9f-4324-af1d-39f24910e058 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:34.571176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a51919-8e9f-4324-af1d-39f24910e058 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:34.571508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41094a-aa38-406b-b7b6-33f608441895 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:34.571899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a51919-8e9f-4324-af1d-39f24910e058 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:34.572100 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41094a-aa38-406b-b7b6-33f608441895 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:34.572208 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a51919-8e9f-4324-af1d-39f24910e058 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:34.572604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41094a-aa38-406b-b7b6-33f608441895 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:34.573558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a51919-8e9f-4324-af1d-39f24910e058 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:34.580538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f41094a-aa38-406b-b7b6-33f608441895 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:34.582042 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f41094a-aa38-406b-b7b6-33f608441895 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:34.592708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 291/561] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:34.592708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a51919-8e9f-4324-af1d-39f24910e058 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:34.595303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-5f41094a-aa38-406b-b7b6-33f608441895 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:34.595303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:34.595303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:34.595303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:34.595303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:34.595303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:34.595303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:34.595303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:34.595303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:34.595303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:34.595303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:34.595303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:34.595303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:20:34.595303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:20:34.595303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:20:34.596273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:20:34.596273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:34.596273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:34.596273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:34.596273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:34.596273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:34.596273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:34.596273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:34.596273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:34.596273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:34.596273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:34.596273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:34.596273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:34.596273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:34.596273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:34.596273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:34.596273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:34.597171 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:20:34.597171 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:20:34.597171 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:20:34.597171 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:20:34.597171 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:20:34.597171 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:20:34.597171 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:34.597171 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:34.597171 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 271/562] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:34.598364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9341a9e-a005-43ac-8333-e7a3816f85cf None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:34.598364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9341a9e-a005-43ac-8333-e7a3816f85cf None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:34.598364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9341a9e-a005-43ac-8333-e7a3816f85cf None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:34.604252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 272/563] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:34.604252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9341a9e-a005-43ac-8333-e7a3816f85cf None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:34.695316 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:34.715546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a51919-8e9f-4324-af1d-39f24910e058 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:34.718839 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9341a9e-a005-43ac-8333-e7a3816f85cf None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:34.752237 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9ac56-cd0e-49d2-9a0f-228ad0e29cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:34.752944 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9ac56-cd0e-49d2-9a0f-228ad0e29cab None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:34.753225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9ac56-cd0e-49d2-9a0f-228ad0e29cab None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:34.753502 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9ac56-cd0e-49d2-9a0f-228ad0e29cab None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:34.754255 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9ac56-cd0e-49d2-9a0f-228ad0e29cab None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:34.754566 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9ac56-cd0e-49d2-9a0f-228ad0e29cab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:34.755959 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9ac56-cd0e-49d2-9a0f-228ad0e29cab None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:34.761629 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4c200-9adf-4538-b8e9-7d2e74f6b442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:34.762314 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4c200-9adf-4538-b8e9-7d2e74f6b442 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:34.762620 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4c200-9adf-4538-b8e9-7d2e74f6b442 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:34.762877 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4c200-9adf-4538-b8e9-7d2e74f6b442 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:34.766262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 292/564] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:20:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:34.766262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9ac56-cd0e-49d2-9a0f-228ad0e29cab None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:34.772345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4c200-9adf-4538-b8e9-7d2e74f6b442 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:34.773859 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4c200-9adf-4538-b8e9-7d2e74f6b442 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:34.787289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-b6f4c200-9adf-4538-b8e9-7d2e74f6b442 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:20:34.787289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:34.787289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:34.787289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:34.787289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:34.787289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:34.787289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:34.787289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:34.787289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:34.787289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:34.787289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:34.787289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:34.787289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:20:34.787289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:20:34.787289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:20:34.787289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:20:34.788538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:34.788538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:34.788538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:20:34.788538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:34.788538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:20:34.788538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:34.788538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:34.788538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:34.788538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:34.788538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:34.788538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:34.788538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:34.788538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:34.788538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:34.788538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:34.788538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:34.788538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:34.788538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:34.789680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:34.789680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:34.789680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:20:34.789680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:20:34.789680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:20:34.789680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:20:34.789680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:20:34.789680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:20:34.789680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:20:34.789680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:34.789680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 273/565] 10.222.0.83 () {56 vars in 1093 bytes} [Mon Aug 1 15:20:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:34.789680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9712944-89dc-45f9-a8d2-68d4017c78a6 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:34.791343 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9712944-89dc-45f9-a8d2-68d4017c78a6 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:34.791343 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9712944-89dc-45f9-a8d2-68d4017c78a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:34.799924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:34.809399 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e9712944-89dc-45f9-a8d2-68d4017c78a6 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:34.810646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e9712944-89dc-45f9-a8d2-68d4017c78a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:34.889382 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 274/566] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:34.890286 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9712944-89dc-45f9-a8d2-68d4017c78a6 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:34.897318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9ac56-cd0e-49d2-9a0f-228ad0e29cab None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:34.934594 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454f15e-55af-410b-b09b-7317e9f3d65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:34.935215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454f15e-55af-410b-b09b-7317e9f3d65d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:34.935519 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454f15e-55af-410b-b09b-7317e9f3d65d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:34.935823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454f15e-55af-410b-b09b-7317e9f3d65d None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:34.936510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454f15e-55af-410b-b09b-7317e9f3d65d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:34.936810 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454f15e-55af-410b-b09b-7317e9f3d65d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:34.938273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454f15e-55af-410b-b09b-7317e9f3d65d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:34.951337 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 293/567] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:20:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:34.952277 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454f15e-55af-410b-b09b-7317e9f3d65d None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:34.970452 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:34.998391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9712944-89dc-45f9-a8d2-68d4017c78a6 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:35.034903 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa2b7e-33f6-4572-85c1-df429ce477d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:35.035478 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa2b7e-33f6-4572-85c1-df429ce477d5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:35.035749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa2b7e-33f6-4572-85c1-df429ce477d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:35.036025 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa2b7e-33f6-4572-85c1-df429ce477d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:35.036157 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:35.047969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aa2b7e-33f6-4572-85c1-df429ce477d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:35.049403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aa2b7e-33f6-4572-85c1-df429ce477d5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:35.070704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454f15e-55af-410b-b09b-7317e9f3d65d None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:35.071211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-00aa2b7e-33f6-4572-85c1-df429ce477d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:35.071211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:35.071211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:35.071211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:35.071211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:35.071211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:35.071211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:35.071211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:35.071211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:35.071211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:35.071211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:35.071211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:35.071211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:35.071211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:35.071211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:35.072637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:35.072637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:35.072637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:35.072637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:35.072637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:35.072637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:35.072637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:35.072637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:35.072637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:35.072637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:35.072637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:35.072637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:35.072637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:35.072637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:35.072637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:35.072637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:35.072637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:35.073533 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:35.073533 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:35.073533 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:35.073533 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:35.073533 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 275/568] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:35.074633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3936450-fe96-41b7-af6f-bb98d56bda45 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:35.074783 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3936450-fe96-41b7-af6f-bb98d56bda45 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:35.075029 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3936450-fe96-41b7-af6f-bb98d56bda45 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:35.082405 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 276/569] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:35.082405 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3936450-fe96-41b7-af6f-bb98d56bda45 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:35.128574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbe778-42c2-4176-a483-5d7d3ba1c502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:35.133471 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbe778-42c2-4176-a483-5d7d3ba1c502 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:35.133752 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbe778-42c2-4176-a483-5d7d3ba1c502 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:35.134910 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbe778-42c2-4176-a483-5d7d3ba1c502 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:35.135793 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fbe778-42c2-4176-a483-5d7d3ba1c502 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:35.136104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fbe778-42c2-4176-a483-5d7d3ba1c502 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:35.137531 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fbe778-42c2-4176-a483-5d7d3ba1c502 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:35.160112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 294/570] 10.222.0.83 () {56 vars in 1133 bytes} [Mon Aug 1 15:20:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:35.160962 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbe778-42c2-4176-a483-5d7d3ba1c502 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:35.192496 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3936450-fe96-41b7-af6f-bb98d56bda45 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:35.220918 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:35.232498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1fa4c4-d03b-4ed3-8972-be03da56dd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:35.233150 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fa4c4-d03b-4ed3-8972-be03da56dd03 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:35.233456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fa4c4-d03b-4ed3-8972-be03da56dd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:35.233725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fa4c4-d03b-4ed3-8972-be03da56dd03 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:35.245065 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1fa4c4-d03b-4ed3-8972-be03da56dd03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:35.250546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-4e1fa4c4-d03b-4ed3-8972-be03da56dd03 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:35.250546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:35.250546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:35.250546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:35.250546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:35.250546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:35.250546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:35.250546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:35.250546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:35.250546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:35.250546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:35.250546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:35.250546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:35.250546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:35.250546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:35.250546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:35.251515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:35.251515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:35.251515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:35.251515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:35.251515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:35.251515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:35.251515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:35.251515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:35.251515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:35.251515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:35.251515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:35.251515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:35.251515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:35.251515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:35.251515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:35.251515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:35.252352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:35.252352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:35.252352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:35.252352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:35.252352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:35.252352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:35.252352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:35.252352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:35.252352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:35.252352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:35.252352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:35.252352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:35.252352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 277/571] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:20:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:35.253042 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1fa4c4-d03b-4ed3-8972-be03da56dd03 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:35.276356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbe778-42c2-4176-a483-5d7d3ba1c502 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:35.310174 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:35.316122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638ed20-673e-42e5-b5f9-b0c35edabe54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:35.316743 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638ed20-673e-42e5-b5f9-b0c35edabe54 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:35.317025 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638ed20-673e-42e5-b5f9-b0c35edabe54 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:35.317300 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638ed20-673e-42e5-b5f9-b0c35edabe54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:35.318121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638ed20-673e-42e5-b5f9-b0c35edabe54 None admin] RBAC: Adding query filter params (role.id=091d39715077464daa4cf890db55e6b9, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=6873a3e440bd4b63b0082fc85beef49c) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:35.318437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638ed20-673e-42e5-b5f9-b0c35edabe54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:35.319880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638ed20-673e-42e5-b5f9-b0c35edabe54 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:35.326437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 295/572] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:20:35 2022] GET /identity/v3/role_assignments?user.id=6873a3e440bd4b63b0082fc85beef49c&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=091d39715077464daa4cf890db55e6b9 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:35.327620 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437dfd2-7b90-45de-921d-231d9d268939 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:35.327904 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437dfd2-7b90-45de-921d-231d9d268939 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:35.328178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437dfd2-7b90-45de-921d-231d9d268939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:35.358825 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2437dfd2-7b90-45de-921d-231d9d268939 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:35.359072 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1fa4c4-d03b-4ed3-8972-be03da56dd03 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:35.360404 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2437dfd2-7b90-45de-921d-231d9d268939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:35.407115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3ef18-5fce-4296-88b1-38763938a02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:35.407967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3ef18-5fce-4296-88b1-38763938a02e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:35.408332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3ef18-5fce-4296-88b1-38763938a02e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:35.408753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3ef18-5fce-4296-88b1-38763938a02e None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:35.409955 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3ef18-5fce-4296-88b1-38763938a02e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:35.411076 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3ef18-5fce-4296-88b1-38763938a02e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:35.412389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3ef18-5fce-4296-88b1-38763938a02e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:35.431480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 278/573] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:35.431480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3ef18-5fce-4296-88b1-38763938a02e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:35.470368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 296/574] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:35.471622 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437dfd2-7b90-45de-921d-231d9d268939 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:35.499204 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:35.578164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3ef18-5fce-4296-88b1-38763938a02e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:35.606111 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437dfd2-7b90-45de-921d-231d9d268939 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:35.625074 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffeec69-330d-4482-b2a2-ad37a4a40ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:35.625754 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffeec69-330d-4482-b2a2-ad37a4a40ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:35.626082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffeec69-330d-4482-b2a2-ad37a4a40ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:35.626347 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffeec69-330d-4482-b2a2-ad37a4a40ce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:35.637408 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:35.639190 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffeec69-330d-4482-b2a2-ad37a4a40ce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:35.640678 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffeec69-330d-4482-b2a2-ad37a4a40ce1 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:35.648267 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665ada4-1d48-45c2-8887-d47f99f26c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:35.648896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665ada4-1d48-45c2-8887-d47f99f26c0e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:35.649175 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665ada4-1d48-45c2-8887-d47f99f26c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:35.649447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665ada4-1d48-45c2-8887-d47f99f26c0e None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:35.650234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665ada4-1d48-45c2-8887-d47f99f26c0e None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:35.650542 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665ada4-1d48-45c2-8887-d47f99f26c0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:35.651908 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665ada4-1d48-45c2-8887-d47f99f26c0e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:35.665923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 297/575] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:20:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:35.666127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-bffeec69-330d-4482-b2a2-ad37a4a40ce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:35.666127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:35.666127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:35.666127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:35.666127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:35.666127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:35.666127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:35.666127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:35.666127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:35.666127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:35.666127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:35.666127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:35.666127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:35.666127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:35.666127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:35.667048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:35.667048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:35.667048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:35.667048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:35.667048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:35.667048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:35.667048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:35.667048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:35.667048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:35.667048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:35.667048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:35.667048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:35.667048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:35.667048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:35.667048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:35.667048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:35.667048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:35.668083 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:35.668083 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:35.668083 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:35.668083 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:35.668308 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665ada4-1d48-45c2-8887-d47f99f26c0e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:35.668393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 279/576] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:35.668393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffeec69-330d-4482-b2a2-ad37a4a40ce1 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:35.789277 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:35.794557 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffeec69-330d-4482-b2a2-ad37a4a40ce1 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:35.798124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665ada4-1d48-45c2-8887-d47f99f26c0e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:35.836280 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64745b76-514f-4670-bb11-781da57f6fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:35.836814 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e6684-927a-4738-9495-15bbbab81cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:35.836977 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64745b76-514f-4670-bb11-781da57f6fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:35.837733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64745b76-514f-4670-bb11-781da57f6fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:35.837931 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64745b76-514f-4670-bb11-781da57f6fe6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:35.838085 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e6684-927a-4738-9495-15bbbab81cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:35.838366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e6684-927a-4738-9495-15bbbab81cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:35.838643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e6684-927a-4738-9495-15bbbab81cf9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:35.846300 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64745b76-514f-4670-bb11-781da57f6fe6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:35.847885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64745b76-514f-4670-bb11-781da57f6fe6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:35.849489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e6684-927a-4738-9495-15bbbab81cf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:35.855857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-b74e6684-927a-4738-9495-15bbbab81cf9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:35.855857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:35.855857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:35.855857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:35.855857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:35.855857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:35.855857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:35.855857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:35.855857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:35.855857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:35.855857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:35.855857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:35.855857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:35.855857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:35.855857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:35.855857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:35.856971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:35.856971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:35.856971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:35.856971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:35.856971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:35.856971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:35.856971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:35.856971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:35.856971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:35.856971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:35.856971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:35.856971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:35.856971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:35.856971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:35.856971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:35.856971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:35.857980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:35.857980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:35.857980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:35.857980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:35.857980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:35.857980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:35.857980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:35.857980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:35.857980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:35.857980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:35.857980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:35.857980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:35.857980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 280/577] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:20:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:35.858769 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e6684-927a-4738-9495-15bbbab81cf9 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:35.868428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-64745b76-514f-4670-bb11-781da57f6fe6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 15:20:35.868428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:35.868428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:35.868428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:35.868428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:35.868428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:35.868428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:35.868428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:35.868428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:35.868428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:35.868428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:35.868428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:35.868428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:20:35.868428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:20:35.868428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:20:35.868428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:20:35.869574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:35.869574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:35.869574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:20:35.869574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:35.869574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:20:35.869574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:35.869574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:35.869574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:35.869574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:35.869574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:35.869574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:35.869574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:35.869574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:35.869574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:35.869574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:35.869574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:35.869574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:35.869574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:35.870848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:35.870848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:35.870848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:20:35.870848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:20:35.870848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:20:35.870848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:20:35.870848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:20:35.870848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:20:35.870848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 15:20:35.870848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:35.873113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 298/578] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:35.874137 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64745b76-514f-4670-bb11-781da57f6fe6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:35.932551 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:35.984534 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e6684-927a-4738-9495-15bbbab81cf9 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:35.993451 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64745b76-514f-4670-bb11-781da57f6fe6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:36.026589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffcb58-b460-4fad-8777-5dbc29236365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:36.027262 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffcb58-b460-4fad-8777-5dbc29236365 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:36.027555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffcb58-b460-4fad-8777-5dbc29236365 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:36.027888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffcb58-b460-4fad-8777-5dbc29236365 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:36.036619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d9992-24a3-4e81-b896-75df5c071163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:36.036619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d9992-24a3-4e81-b896-75df5c071163 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:36.037288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d9992-24a3-4e81-b896-75df5c071163 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:36.037288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d9992-24a3-4e81-b896-75df5c071163 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:36.037917 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d9992-24a3-4e81-b896-75df5c071163 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:36.038172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d9992-24a3-4e81-b896-75df5c071163 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:36.039402 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d9992-24a3-4e81-b896-75df5c071163 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:36.042660 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ffcb58-b460-4fad-8777-5dbc29236365 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:36.044150 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ffcb58-b460-4fad-8777-5dbc29236365 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:36.062530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 299/579] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:36.062690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-b1ffcb58-b460-4fad-8777-5dbc29236365 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:36.062690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:36.062690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:36.062690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:36.062690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:36.062690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:36.062690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:36.062690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:36.062690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:36.062690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:36.062690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:36.062690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:36.062690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:36.062690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:36.062690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:36.064026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:36.064026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:36.064026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:36.064026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:36.064026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:36.064026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:36.064026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:36.064026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:36.064026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:36.064026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:36.064026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:36.064026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:36.064026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:36.064026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:36.064026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:36.064026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:36.064026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:36.065091 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:36.065091 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:36.065091 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:36.065091 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:36.065381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d9992-24a3-4e81-b896-75df5c071163 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:36.065475 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 281/580] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:36.065475 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffcb58-b460-4fad-8777-5dbc29236365 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:36.083955 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:36.192423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffcb58-b460-4fad-8777-5dbc29236365 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:36.205627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d9992-24a3-4e81-b896-75df5c071163 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:36.216528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:36.235274 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f914e09-068a-44b0-b32b-e9dd8a000b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:36.236093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f914e09-068a-44b0-b32b-e9dd8a000b0d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:36.236484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f914e09-068a-44b0-b32b-e9dd8a000b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:36.236764 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f914e09-068a-44b0-b32b-e9dd8a000b0d None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:36.237560 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f914e09-068a-44b0-b32b-e9dd8a000b0d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:36.237872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f914e09-068a-44b0-b32b-e9dd8a000b0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:36.239604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f914e09-068a-44b0-b32b-e9dd8a000b0d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:36.244857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa004b-a688-4f0e-90bc-b6e9faae4ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:36.245480 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa004b-a688-4f0e-90bc-b6e9faae4ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:36.245762 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa004b-a688-4f0e-90bc-b6e9faae4ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:36.246073 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa004b-a688-4f0e-90bc-b6e9faae4ba0 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:36.246799 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa004b-a688-4f0e-90bc-b6e9faae4ba0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:36.247266 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa004b-a688-4f0e-90bc-b6e9faae4ba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:36.248632 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa004b-a688-4f0e-90bc-b6e9faae4ba0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:36.257584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 300/581] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:20:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:36.258443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa004b-a688-4f0e-90bc-b6e9faae4ba0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:36.263662 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 282/582] 10.222.0.83 () {56 vars in 1137 bytes} [Mon Aug 1 15:20:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:36.264732 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f914e09-068a-44b0-b32b-e9dd8a000b0d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:36.336901 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:36.380025 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f914e09-068a-44b0-b32b-e9dd8a000b0d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:36.390461 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa004b-a688-4f0e-90bc-b6e9faae4ba0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:36.422646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c846542-8c03-4e7e-bffa-a8c76174b7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:36.423587 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c846542-8c03-4e7e-bffa-a8c76174b7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:36.423947 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c846542-8c03-4e7e-bffa-a8c76174b7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:36.424236 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c846542-8c03-4e7e-bffa-a8c76174b7b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:36.424924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c846542-8c03-4e7e-bffa-a8c76174b7b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:36.425232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c846542-8c03-4e7e-bffa-a8c76174b7b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:36.427009 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c846542-8c03-4e7e-bffa-a8c76174b7b8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:36.432388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ad357-109f-4167-8dd3-5d79a5378429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:36.433086 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ad357-109f-4167-8dd3-5d79a5378429 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:36.433315 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ad357-109f-4167-8dd3-5d79a5378429 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:36.433584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ad357-109f-4167-8dd3-5d79a5378429 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:36.443997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 283/583] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:36.443997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c846542-8c03-4e7e-bffa-a8c76174b7b8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:36.446207 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ad357-109f-4167-8dd3-5d79a5378429 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:36.448191 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ad357-109f-4167-8dd3-5d79a5378429 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:36.460634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-a53ad357-109f-4167-8dd3-5d79a5378429 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:36.460634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:36.460634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:36.460634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:36.460634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:36.460634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:36.460634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:36.460634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:36.460634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:36.460634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:36.460634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:36.460634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:36.460634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:20:36.460634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:20:36.460634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:20:36.461650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:20:36.461650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:36.461650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:36.461650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:36.461650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:36.461650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:36.461650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:36.461650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:36.461650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:36.461650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:36.461650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:36.461650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:36.461650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:36.461650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:36.461650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:36.461650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:36.461650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:36.468524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:20:36.468524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:20:36.468524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:20:36.468524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:20:36.468524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:20:36.468524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:20:36.468524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:36.468524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:36.468524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 301/584] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:36.468524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ad357-109f-4167-8dd3-5d79a5378429 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:36.506405 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:36.581297 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c846542-8c03-4e7e-bffa-a8c76174b7b8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:36.592579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ad357-109f-4167-8dd3-5d79a5378429 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:36.621709 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:36.630369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd4463-f0c7-4b96-b772-ed2c097dbade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:36.631014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd4463-f0c7-4b96-b772-ed2c097dbade None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:36.631344 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd4463-f0c7-4b96-b772-ed2c097dbade None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:36.631556 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd4463-f0c7-4b96-b772-ed2c097dbade None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:36.633365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52377643-7e9a-483c-9f52-cf5b8a3fe9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:36.635082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52377643-7e9a-483c-9f52-cf5b8a3fe9b4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:36.635522 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52377643-7e9a-483c-9f52-cf5b8a3fe9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:36.635944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52377643-7e9a-483c-9f52-cf5b8a3fe9b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:36.644476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd4463-f0c7-4b96-b772-ed2c097dbade None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:36.646133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd4463-f0c7-4b96-b772-ed2c097dbade None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:36.653795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52377643-7e9a-483c-9f52-cf5b8a3fe9b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:36.656148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52377643-7e9a-483c-9f52-cf5b8a3fe9b4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:36.663835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-38fd4463-f0c7-4b96-b772-ed2c097dbade None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:36.663835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:36.663835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:36.663835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:36.663835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:36.663835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:36.663835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:36.663835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:36.663835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:36.663835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:36.663835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:36.663835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:36.663835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:36.663835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:36.663835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:36.664885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:36.664885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:36.664885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:36.664885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:36.664885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:36.664885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:36.664885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:36.664885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:36.664885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:36.664885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:36.664885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:36.664885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:36.664885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:36.664885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:36.664885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:36.664885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:36.664885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:36.666068 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:36.666068 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:36.666068 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:36.666068 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:36.667881 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 302/585] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:36.668952 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd4463-f0c7-4b96-b772-ed2c097dbade None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:36.681433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-52377643-7e9a-483c-9f52-cf5b8a3fe9b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:36.681433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:36.681433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:36.681433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:36.681433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:36.681433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:36.681433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:36.681433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:36.681433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:36.681433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:36.681433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:36.681433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:36.681433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:36.681433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:36.681433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:36.682420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:36.682420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:36.682420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:36.682420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:36.682420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:36.682420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:36.682420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:36.682420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:36.682420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:36.682420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:36.682420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:36.682420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:36.682420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:36.682420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:36.682420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:36.682420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:36.682420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:36.683335 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:36.683335 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:36.683335 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:36.683335 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:36.683335 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 284/586] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:36.684048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52377643-7e9a-483c-9f52-cf5b8a3fe9b4 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:36.770122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:36.794814 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd4463-f0c7-4b96-b772-ed2c097dbade None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:36.808240 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52377643-7e9a-483c-9f52-cf5b8a3fe9b4 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:36.836259 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a264ee-d67b-4f94-8d75-725295fba7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:36.836904 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a264ee-d67b-4f94-8d75-725295fba7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:36.837253 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a264ee-d67b-4f94-8d75-725295fba7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:36.837485 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a264ee-d67b-4f94-8d75-725295fba7f4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:36.845561 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a264ee-d67b-4f94-8d75-725295fba7f4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:36.847100 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a264ee-d67b-4f94-8d75-725295fba7f4 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:36.847421 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72015b-c715-4273-8fa7-c991fd1c1990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:36.848050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72015b-c715-4273-8fa7-c991fd1c1990 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:36.848337 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72015b-c715-4273-8fa7-c991fd1c1990 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:36.848615 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72015b-c715-4273-8fa7-c991fd1c1990 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:36.849378 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72015b-c715-4273-8fa7-c991fd1c1990 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:36.849692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72015b-c715-4273-8fa7-c991fd1c1990 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:36.851231 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72015b-c715-4273-8fa7-c991fd1c1990 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:36.859780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-17a264ee-d67b-4f94-8d75-725295fba7f4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:20:36.859780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:36.859780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:36.859780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:36.859780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:36.859780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:36.859780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:36.859780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:36.859780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:36.859780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:36.859780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:36.859780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:36.859780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:20:36.859780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:20:36.859780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:20:36.859780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:20:36.860966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:36.860966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:36.860966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:20:36.860966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:36.860966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:20:36.860966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:36.860966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:36.860966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:36.860966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:36.860966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:36.860966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:36.860966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:36.860966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:36.860966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:36.860966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:36.860966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:36.860966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:36.860966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:36.862173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:36.862173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:36.862173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:20:36.862173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:20:36.862173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:20:36.862173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:20:36.862173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:20:36.862173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:20:36.862173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:20:36.862173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:36.862173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 303/587] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:36.862173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a264ee-d67b-4f94-8d75-725295fba7f4 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:36.869296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 285/588] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:20:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:36.870883 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72015b-c715-4273-8fa7-c991fd1c1990 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:36.926666 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:36.989443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a264ee-d67b-4f94-8d75-725295fba7f4 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:36.992662 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72015b-c715-4273-8fa7-c991fd1c1990 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:37.028588 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc0da4-4593-44b6-b291-dc5ae5242e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:37.029129 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc0da4-4593-44b6-b291-dc5ae5242e6d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:37.029415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc0da4-4593-44b6-b291-dc5ae5242e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:37.029695 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc0da4-4593-44b6-b291-dc5ae5242e6d None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:37.030425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc0da4-4593-44b6-b291-dc5ae5242e6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:37.030733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc0da4-4593-44b6-b291-dc5ae5242e6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:37.032127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc0da4-4593-44b6-b291-dc5ae5242e6d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:37.034098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e7145-2200-4966-a375-2e20bcfd4e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:37.035369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e7145-2200-4966-a375-2e20bcfd4e8f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:37.035850 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e7145-2200-4966-a375-2e20bcfd4e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:37.036267 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e7145-2200-4966-a375-2e20bcfd4e8f None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:37.037105 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e7145-2200-4966-a375-2e20bcfd4e8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:37.037582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e7145-2200-4966-a375-2e20bcfd4e8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:37.039627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e7145-2200-4966-a375-2e20bcfd4e8f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:37.044273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 286/589] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:37.045108 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc0da4-4593-44b6-b291-dc5ae5242e6d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:37.061324 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 304/590] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:37.061324 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e7145-2200-4966-a375-2e20bcfd4e8f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:37.065505 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:37.180329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e7145-2200-4966-a375-2e20bcfd4e8f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:37.183194 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:37.189416 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc0da4-4593-44b6-b291-dc5ae5242e6d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:37.226365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b8fc0-492e-491c-b0c5-5ea8422d54db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:37.227059 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b8fc0-492e-491c-b0c5-5ea8422d54db None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:37.227343 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b8fc0-492e-491c-b0c5-5ea8422d54db None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:37.227621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b8fc0-492e-491c-b0c5-5ea8422d54db None admin] PATH_INFO: `/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/25dc41ffc41c47fdbeec06110d04610c/roles/091d39715077464daa4cf890db55e6b9` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:37.237159 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6dbf39-ea28-4ab3-8999-f649b3ee7f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:37.237742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dbf39-ea28-4ab3-8999-f649b3ee7f69 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:37.237830 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dbf39-ea28-4ab3-8999-f649b3ee7f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:37.238145 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dbf39-ea28-4ab3-8999-f649b3ee7f69 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:37.238948 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6dbf39-ea28-4ab3-8999-f649b3ee7f69 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:37.239251 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6dbf39-ea28-4ab3-8999-f649b3ee7f69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:37.240781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6dbf39-ea28-4ab3-8999-f649b3ee7f69 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:37.248202 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b8fc0-492e-491c-b0c5-5ea8422d54db None admin] RBAC: Authorizing `identity:create_grant(project_id=1f4e27cc50bd4f9f8ced4cc544ab3064, user_id=25dc41ffc41c47fdbeec06110d04610c, role_id=091d39715077464daa4cf890db55e6b9)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:37.249765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b8fc0-492e-491c-b0c5-5ea8422d54db None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:37.260671 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 287/591] 10.222.0.83 () {56 vars in 1137 bytes} [Mon Aug 1 15:20:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:37.261544 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6dbf39-ea28-4ab3-8999-f649b3ee7f69 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:37.270056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 305/592] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:20:37 2022] PUT /identity/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/25dc41ffc41c47fdbeec06110d04610c/roles/091d39715077464daa4cf890db55e6b9 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:20:37.270929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b8fc0-492e-491c-b0c5-5ea8422d54db None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:37.369596 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:37.444092 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b8fc0-492e-491c-b0c5-5ea8422d54db None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:37.465618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6dbf39-ea28-4ab3-8999-f649b3ee7f69 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:37.494054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8785199-7139-4077-ba85-afcea4a8eb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:37.494897 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8785199-7139-4077-ba85-afcea4a8eb7e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:37.495166 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8785199-7139-4077-ba85-afcea4a8eb7e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:37.495447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8785199-7139-4077-ba85-afcea4a8eb7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:37.503290 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8785199-7139-4077-ba85-afcea4a8eb7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:37.504751 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8785199-7139-4077-ba85-afcea4a8eb7e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:37.505800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237db7f-ac35-4dc2-b3e5-d39df6472c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:37.506547 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237db7f-ac35-4dc2-b3e5-d39df6472c91 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:37.506814 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237db7f-ac35-4dc2-b3e5-d39df6472c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:37.507087 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237db7f-ac35-4dc2-b3e5-d39df6472c91 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:37.517960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-d8785199-7139-4077-ba85-afcea4a8eb7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:37.517960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:37.517960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:37.517960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:37.517960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:37.517960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:37.517960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:37.517960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:37.517960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:37.517960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:37.517960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:37.517960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:37.517960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:20:37.517960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:20:37.517960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:20:37.518996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:20:37.518996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:37.518996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:37.518996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:37.518996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:37.518996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:37.518996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:37.518996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:37.518996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:37.518996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:37.518996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:37.518996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:37.518996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:37.518996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:37.518996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:37.518996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:37.518996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:37.520113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:20:37.520113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:20:37.520113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:20:37.520113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:20:37.520113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:20:37.520113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:20:37.520113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:37.520113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:37.520657 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237db7f-ac35-4dc2-b3e5-d39df6472c91 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:37.520657 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237db7f-ac35-4dc2-b3e5-d39df6472c91 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:37.522177 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 306/593] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:37.523241 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8785199-7139-4077-ba85-afcea4a8eb7e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:37.545348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-6237db7f-ac35-4dc2-b3e5-d39df6472c91 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 15:20:37.545348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:37.545348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:37.545348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:37.545348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:37.545348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:37.545348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:37.545348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:37.545348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:37.545348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:37.545348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:37.545348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:37.545348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:20:37.545348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:20:37.545348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:20:37.545348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:20:37.546700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:37.546700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:37.546700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:20:37.546700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:37.546700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:20:37.546700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:37.546700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:37.546700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:37.546700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:37.546700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:37.546700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:37.546700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:37.546700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:37.546700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:37.546700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:37.546700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:37.546700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:37.546700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:37.548039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:37.548039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:37.548039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:20:37.548039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:20:37.548039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:20:37.548039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:20:37.548039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:20:37.548039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:20:37.548039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 15:20:37.548039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:37.551356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 288/594] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:20:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:37.552693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e94db-cb3b-4fcc-83cc-6f2f71029ad2 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:37.552978 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e94db-cb3b-4fcc-83cc-6f2f71029ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:37.553252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e94db-cb3b-4fcc-83cc-6f2f71029ad2 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:37.558306 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 289/595] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:37.560076 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e94db-cb3b-4fcc-83cc-6f2f71029ad2 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:37.591702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:37.665636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8785199-7139-4077-ba85-afcea4a8eb7e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:37.682286 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e94db-cb3b-4fcc-83cc-6f2f71029ad2 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:37.683734 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:37.707671 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63eb9a0-f2a6-4678-a613-49e6e76cb605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:37.708611 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63eb9a0-f2a6-4678-a613-49e6e76cb605 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:37.709225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63eb9a0-f2a6-4678-a613-49e6e76cb605 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:37.709770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63eb9a0-f2a6-4678-a613-49e6e76cb605 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:37.723322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63eb9a0-f2a6-4678-a613-49e6e76cb605 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:37.725555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63eb9a0-f2a6-4678-a613-49e6e76cb605 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:37.725690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736ad5c-5ea0-4e40-916d-5087b9545773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:37.726343 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736ad5c-5ea0-4e40-916d-5087b9545773 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:37.726638 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736ad5c-5ea0-4e40-916d-5087b9545773 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:37.726899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736ad5c-5ea0-4e40-916d-5087b9545773 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:37.727693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736ad5c-5ea0-4e40-916d-5087b9545773 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:37.728082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736ad5c-5ea0-4e40-916d-5087b9545773 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:37.730237 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736ad5c-5ea0-4e40-916d-5087b9545773 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:37.744258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-a63eb9a0-f2a6-4678-a613-49e6e76cb605 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:37.744258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:37.744258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:37.744258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:37.744258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:37.744258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:37.744258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:37.744258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:37.744258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:37.744258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:37.744258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:37.744258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:37.744258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:37.744258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:37.744258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:37.745434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:37.745434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:37.745434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:37.745434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:37.745434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:37.745434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:37.745434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:37.745434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:37.745434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:37.745434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:37.745434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:37.745434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:37.745434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:37.745434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:37.745434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:37.745434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:37.745434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:37.746581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:37.746581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:37.746581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:37.746581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:37.746581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 307/596] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:37.746921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63eb9a0-f2a6-4678-a613-49e6e76cb605 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:37.754577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 290/597] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:20:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:37.755798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906de02c-e079-4b4c-86d9-db4c8d09b527 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:37.756090 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906de02c-e079-4b4c-86d9-db4c8d09b527 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:37.756362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906de02c-e079-4b4c-86d9-db4c8d09b527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:37.777002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-906de02c-e079-4b4c-86d9-db4c8d09b527 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:37.778305 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-906de02c-e079-4b4c-86d9-db4c8d09b527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:37.880662 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 291/598] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:37.882172 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906de02c-e079-4b4c-86d9-db4c8d09b527 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:37.899145 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63eb9a0-f2a6-4678-a613-49e6e76cb605 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:37.901251 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:37.941796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75959de-7752-448c-ab0f-ef2dd1276ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:37.943308 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75959de-7752-448c-ab0f-ef2dd1276ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:37.944116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75959de-7752-448c-ab0f-ef2dd1276ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:37.944607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75959de-7752-448c-ab0f-ef2dd1276ee3 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:37.945837 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75959de-7752-448c-ab0f-ef2dd1276ee3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:37.946348 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75959de-7752-448c-ab0f-ef2dd1276ee3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:37.948473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75959de-7752-448c-ab0f-ef2dd1276ee3 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:37.968385 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 308/599] 10.222.0.83 () {56 vars in 1139 bytes} [Mon Aug 1 15:20:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:37.969659 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75959de-7752-448c-ab0f-ef2dd1276ee3 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:37.996430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:38.020832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906de02c-e079-4b4c-86d9-db4c8d09b527 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:38.073272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a972c-3ce3-4ec0-9dc3-ebb8e393ec88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:38.074122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a972c-3ce3-4ec0-9dc3-ebb8e393ec88 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:38.074433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a972c-3ce3-4ec0-9dc3-ebb8e393ec88 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:38.074711 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a972c-3ce3-4ec0-9dc3-ebb8e393ec88 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:38.075500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a972c-3ce3-4ec0-9dc3-ebb8e393ec88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:38.075866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a972c-3ce3-4ec0-9dc3-ebb8e393ec88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:38.077400 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a972c-3ce3-4ec0-9dc3-ebb8e393ec88 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:38.087700 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75959de-7752-448c-ab0f-ef2dd1276ee3 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:38.093182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 292/600] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:38.094353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a972c-3ce3-4ec0-9dc3-ebb8e393ec88 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:38.131897 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac472e-5b5a-4a64-a142-fb495d4f0144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:38.132329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac472e-5b5a-4a64-a142-fb495d4f0144 None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:38.132607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac472e-5b5a-4a64-a142-fb495d4f0144 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:38.132877 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac472e-5b5a-4a64-a142-fb495d4f0144 None admin] PATH_INFO: `/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/69d216a568b546589e854c2b4476efe6/roles/091d39715077464daa4cf890db55e6b9` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:38.152373 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac472e-5b5a-4a64-a142-fb495d4f0144 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f4e27cc50bd4f9f8ced4cc544ab3064, user_id=69d216a568b546589e854c2b4476efe6, role_id=091d39715077464daa4cf890db55e6b9)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:38.154233 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac472e-5b5a-4a64-a142-fb495d4f0144 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:38.177929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 309/601] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:20:37 2022] PUT /identity/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/69d216a568b546589e854c2b4476efe6/roles/091d39715077464daa4cf890db55e6b9 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:20:38.177929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac472e-5b5a-4a64-a142-fb495d4f0144 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:38.187781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:38.243948 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a972c-3ce3-4ec0-9dc3-ebb8e393ec88 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:38.286048 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:38.296436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c71996-9905-4902-81b9-5d1ba28d8d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:38.297103 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c71996-9905-4902-81b9-5d1ba28d8d05 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:38.297386 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c71996-9905-4902-81b9-5d1ba28d8d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:38.297671 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c71996-9905-4902-81b9-5d1ba28d8d05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:38.315079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c71996-9905-4902-81b9-5d1ba28d8d05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:38.323566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-b6c71996-9905-4902-81b9-5d1ba28d8d05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:38.323566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:38.323566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:38.323566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:38.323566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:38.323566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:38.323566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:38.323566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:38.323566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:38.323566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:38.323566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:38.323566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:38.323566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:38.323566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:38.323566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:38.323566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:38.324586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:38.324586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:38.324586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:38.324586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:38.324586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:38.324586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:38.324586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:38.324586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:38.324586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:38.324586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:38.324586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:38.324586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:38.324586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:38.324586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:38.324586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:38.324586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:38.325444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:38.325444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:38.325444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:38.325444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:38.325444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:38.325444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:38.325444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:38.325444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:38.325444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:38.325444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:38.325444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:38.325444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:38.326118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 293/602] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:20:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:38.326733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c71996-9905-4902-81b9-5d1ba28d8d05 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:38.338570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac472e-5b5a-4a64-a142-fb495d4f0144 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:38.391409 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb8664-addd-4765-a669-27aeba0fd099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:38.392899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb8664-addd-4765-a669-27aeba0fd099 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:38.393305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb8664-addd-4765-a669-27aeba0fd099 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:38.393588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb8664-addd-4765-a669-27aeba0fd099 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:38.408465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb8664-addd-4765-a669-27aeba0fd099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:38.410802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb8664-addd-4765-a669-27aeba0fd099 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:38.434037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-83bb8664-addd-4765-a669-27aeba0fd099 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:38.434037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:38.434037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:38.434037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:38.434037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:38.434037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:38.434037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:38.434037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:38.434037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:38.434037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:38.434037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:38.434037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:38.434037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:38.434037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:38.434037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:38.435149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:38.435149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:38.435149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:38.435149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:38.435149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:38.435149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:38.435149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:38.435149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:38.435149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:38.435149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:38.435149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:38.435149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:38.435149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:38.435149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:38.435149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:38.435149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:38.435149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:38.436122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:38.436122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:38.436122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:38.436122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:38.440226 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 310/603] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:38.442078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb8664-addd-4765-a669-27aeba0fd099 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:38.479030 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c71996-9905-4902-81b9-5d1ba28d8d05 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:38.517565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:38.528743 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74169398-eca1-40eb-be9f-b2c436fff8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:38.530621 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74169398-eca1-40eb-be9f-b2c436fff8fe None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:38.531111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74169398-eca1-40eb-be9f-b2c436fff8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:38.531842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74169398-eca1-40eb-be9f-b2c436fff8fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:38.541766 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74169398-eca1-40eb-be9f-b2c436fff8fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:38.544305 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74169398-eca1-40eb-be9f-b2c436fff8fe None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:38.561733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-74169398-eca1-40eb-be9f-b2c436fff8fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:38.561733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:38.561733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:38.561733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:38.561733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:38.561733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:38.561733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:38.561733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:38.561733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:38.561733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:38.561733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:38.561733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:38.561733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:20:38.561733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:20:38.561733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:20:38.562873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:20:38.562873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:38.562873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:38.562873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:38.562873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:38.562873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:38.562873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:38.562873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:38.562873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:38.562873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:38.562873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:38.562873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:38.562873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:38.562873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:38.562873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:38.562873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:38.562873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:38.563905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:20:38.563905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:20:38.563905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:20:38.563905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:20:38.563905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:20:38.563905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:20:38.563905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:38.563905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:38.566354 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 294/604] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:38.567699 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74169398-eca1-40eb-be9f-b2c436fff8fe None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:38.590642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb8664-addd-4765-a669-27aeba0fd099 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:38.627963 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:38.643354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b6ca6-f735-45aa-aed4-64441b524781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:38.644327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b6ca6-f735-45aa-aed4-64441b524781 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:38.644726 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b6ca6-f735-45aa-aed4-64441b524781 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:38.645140 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b6ca6-f735-45aa-aed4-64441b524781 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:38.646596 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b6ca6-f735-45aa-aed4-64441b524781 None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:38.647076 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b6ca6-f735-45aa-aed4-64441b524781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:38.649043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b6ca6-f735-45aa-aed4-64441b524781 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:38.665989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 311/605] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:20:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:38.667354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b6ca6-f735-45aa-aed4-64441b524781 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:38.713466 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74169398-eca1-40eb-be9f-b2c436fff8fe None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:38.750729 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e15284-a0da-4283-bc3d-601bf424e184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:38.751628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e15284-a0da-4283-bc3d-601bf424e184 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:38.751861 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e15284-a0da-4283-bc3d-601bf424e184 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:38.752208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e15284-a0da-4283-bc3d-601bf424e184 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:38.752980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e15284-a0da-4283-bc3d-601bf424e184 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:38.753376 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e15284-a0da-4283-bc3d-601bf424e184 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:38.754895 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e15284-a0da-4283-bc3d-601bf424e184 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:38.772349 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 295/606] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:38.773294 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e15284-a0da-4283-bc3d-601bf424e184 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:38.811416 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b6ca6-f735-45aa-aed4-64441b524781 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:38.830472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:38.854017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719744e-0e90-4259-bbfe-d314f970c32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:38.855429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719744e-0e90-4259-bbfe-d314f970c32a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:38.856027 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719744e-0e90-4259-bbfe-d314f970c32a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:38.856959 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719744e-0e90-4259-bbfe-d314f970c32a None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:38.858293 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719744e-0e90-4259-bbfe-d314f970c32a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:38.859232 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719744e-0e90-4259-bbfe-d314f970c32a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:38.862237 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719744e-0e90-4259-bbfe-d314f970c32a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:38.878116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 312/607] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:20:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:38.879552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719744e-0e90-4259-bbfe-d314f970c32a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:38.885354 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e15284-a0da-4283-bc3d-601bf424e184 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:38.930069 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c4fd1-4dc6-4c5e-96e9-8a6ca22acd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:38.931582 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c4fd1-4dc6-4c5e-96e9-8a6ca22acd2e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:38.932375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c4fd1-4dc6-4c5e-96e9-8a6ca22acd2e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:38.932859 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c4fd1-4dc6-4c5e-96e9-8a6ca22acd2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:38.935245 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:38.950818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c4fd1-4dc6-4c5e-96e9-8a6ca22acd2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:38.953228 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c4fd1-4dc6-4c5e-96e9-8a6ca22acd2e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:38.974977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-f50c4fd1-4dc6-4c5e-96e9-8a6ca22acd2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:38.974977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:38.974977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:38.974977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:38.974977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:38.974977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:38.974977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:38.974977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:38.974977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:38.974977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:38.974977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:38.974977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:38.974977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:38.974977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:38.974977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:38.975933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:38.975933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:38.975933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:38.975933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:38.975933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:38.975933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:38.975933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:38.975933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:38.975933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:38.975933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:38.975933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:38.975933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:38.975933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:38.975933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:38.975933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:38.975933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:38.975933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:38.976903 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:38.976903 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:38.976903 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:38.976903 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:38.984659 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 296/608] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:38.984659 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c4fd1-4dc6-4c5e-96e9-8a6ca22acd2e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:39.010423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719744e-0e90-4259-bbfe-d314f970c32a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:39.059673 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c553c-8f65-4f69-a4ae-9ce2bea892f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:39.060345 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c553c-8f65-4f69-a4ae-9ce2bea892f3 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:39.060631 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c553c-8f65-4f69-a4ae-9ce2bea892f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:39.060904 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c553c-8f65-4f69-a4ae-9ce2bea892f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:39.070692 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c553c-8f65-4f69-a4ae-9ce2bea892f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:39.072816 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c553c-8f65-4f69-a4ae-9ce2bea892f3 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:39.086820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-534c553c-8f65-4f69-a4ae-9ce2bea892f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:39.086820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:39.086820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:39.086820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:39.086820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:39.086820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:39.086820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:39.086820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:39.086820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:39.086820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:39.086820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:39.086820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:39.086820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:20:39.086820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:20:39.086820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:20:39.088291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:20:39.088291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:39.088291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:39.088291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:39.088291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:39.088291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:39.088291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:39.088291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:39.088291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:39.088291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:39.088291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:39.088291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:39.088291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:39.088291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:39.088291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:39.088291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:39.088291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:39.089177 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:20:39.089177 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:20:39.089177 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:20:39.089177 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:20:39.089177 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:20:39.089177 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:20:39.089177 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:39.089177 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:39.093671 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 313/609] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:39.094585 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c553c-8f65-4f69-a4ae-9ce2bea892f3 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:39.127091 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:39.134361 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c4fd1-4dc6-4c5e-96e9-8a6ca22acd2e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:39.183230 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6598391-88ea-4989-aa10-b0d53c509cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:39.184288 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6598391-88ea-4989-aa10-b0d53c509cd7 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:39.184777 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6598391-88ea-4989-aa10-b0d53c509cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:39.185275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6598391-88ea-4989-aa10-b0d53c509cd7 None admin] PATH_INFO: `/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/f54b570990794801a5e45918845569b8/roles/091d39715077464daa4cf890db55e6b9` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:39.205777 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6598391-88ea-4989-aa10-b0d53c509cd7 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f4e27cc50bd4f9f8ced4cc544ab3064, user_id=f54b570990794801a5e45918845569b8, role_id=091d39715077464daa4cf890db55e6b9)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:39.208517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6598391-88ea-4989-aa10-b0d53c509cd7 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:39.214067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c553c-8f65-4f69-a4ae-9ce2bea892f3 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:39.224582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:39.244557 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 297/610] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:20:38 2022] PUT /identity/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/f54b570990794801a5e45918845569b8/roles/091d39715077464daa4cf890db55e6b9 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:20:39.245711 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6598391-88ea-4989-aa10-b0d53c509cd7 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:39.263421 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ebadd-6933-4d87-a71e-738e80842a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:39.264162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ebadd-6933-4d87-a71e-738e80842a64 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:39.264484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ebadd-6933-4d87-a71e-738e80842a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:39.264875 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ebadd-6933-4d87-a71e-738e80842a64 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:39.265651 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ebadd-6933-4d87-a71e-738e80842a64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:39.266075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ebadd-6933-4d87-a71e-738e80842a64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:39.267630 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ebadd-6933-4d87-a71e-738e80842a64 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:39.284880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 314/611] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:39.290987 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ebadd-6933-4d87-a71e-738e80842a64 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:39.430960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ebadd-6933-4d87-a71e-738e80842a64 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:39.488687 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4895c-71a3-4e31-81fd-fa746f20c40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:39.490487 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4895c-71a3-4e31-81fd-fa746f20c40f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:39.490943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4895c-71a3-4e31-81fd-fa746f20c40f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:39.491356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4895c-71a3-4e31-81fd-fa746f20c40f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:39.516352 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b4895c-71a3-4e31-81fd-fa746f20c40f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:39.518191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6598391-88ea-4989-aa10-b0d53c509cd7 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:39.518826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b4895c-71a3-4e31-81fd-fa746f20c40f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:39.540031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-66b4895c-71a3-4e31-81fd-fa746f20c40f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:20:39.540031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:39.540031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:39.540031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:39.540031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:39.540031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:39.540031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:39.540031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:39.540031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:39.540031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:39.540031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:39.540031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:39.540031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:20:39.540031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:20:39.540031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:20:39.540031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:20:39.541085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:39.541085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:39.541085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:20:39.541085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:39.541085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:20:39.541085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:39.541085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:39.541085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:39.541085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:39.541085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:39.541085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:39.541085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:39.541085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:39.541085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:39.541085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:39.541085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:39.541085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:39.541085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:39.542060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:39.542060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:39.542060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:20:39.542060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:20:39.542060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:20:39.542060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:20:39.542060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:20:39.542060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:20:39.542060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:20:39.542060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:39.542060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 315/612] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:20:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:20:39.549381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4895c-71a3-4e31-81fd-fa746f20c40f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:39.564207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1137e-1f87-4301-ba53-45e456d07b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:39.564901 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1137e-1f87-4301-ba53-45e456d07b3a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:39.565190 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1137e-1f87-4301-ba53-45e456d07b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:39.565463 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1137e-1f87-4301-ba53-45e456d07b3a None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:39.566268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1137e-1f87-4301-ba53-45e456d07b3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:39.566576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1137e-1f87-4301-ba53-45e456d07b3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:39.568047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1137e-1f87-4301-ba53-45e456d07b3a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:39.585448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 298/613] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:20:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:39.586800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1849fa9-1d18-43b8-af74-0a15d1bae1b2 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:39.587138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1849fa9-1d18-43b8-af74-0a15d1bae1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:39.587386 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1849fa9-1d18-43b8-af74-0a15d1bae1b2 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:39.588901 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 299/614] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:39.591140 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1849fa9-1d18-43b8-af74-0a15d1bae1b2 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:39.595961 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:39.614599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:39.680398 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4895c-71a3-4e31-81fd-fa746f20c40f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:39.721118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58940b17-8dfe-4368-b763-a78fc1d3147c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:39.721721 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58940b17-8dfe-4368-b763-a78fc1d3147c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:39.722088 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58940b17-8dfe-4368-b763-a78fc1d3147c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:39.722367 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58940b17-8dfe-4368-b763-a78fc1d3147c None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:39.723245 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58940b17-8dfe-4368-b763-a78fc1d3147c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:39.723581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58940b17-8dfe-4368-b763-a78fc1d3147c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:39.725363 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58940b17-8dfe-4368-b763-a78fc1d3147c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:39.741269 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1849fa9-1d18-43b8-af74-0a15d1bae1b2 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:39.746932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 316/615] 10.222.0.83 () {56 vars in 1135 bytes} [Mon Aug 1 15:20:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:39.748458 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e547da8-3d9b-49b3-bf65-3dc42fb0dc93 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:39.748798 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e547da8-3d9b-49b3-bf65-3dc42fb0dc93 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:39.749078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e547da8-3d9b-49b3-bf65-3dc42fb0dc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:39.770273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1e547da8-3d9b-49b3-bf65-3dc42fb0dc93 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:39.771637 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1e547da8-3d9b-49b3-bf65-3dc42fb0dc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:39.789494 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6328f-6c89-47e6-8737-cfc20ae7d3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:39.790979 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6328f-6c89-47e6-8737-cfc20ae7d3ca None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:39.791759 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6328f-6c89-47e6-8737-cfc20ae7d3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:39.792257 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6328f-6c89-47e6-8737-cfc20ae7d3ca None admin] PATH_INFO: `/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/76311c7173b442c0af2c745e3e70a07d/roles/091d39715077464daa4cf890db55e6b9` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:39.813622 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6328f-6c89-47e6-8737-cfc20ae7d3ca None admin] RBAC: Authorizing `identity:create_grant(project_id=1f4e27cc50bd4f9f8ced4cc544ab3064, user_id=76311c7173b442c0af2c745e3e70a07d, role_id=091d39715077464daa4cf890db55e6b9)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:39.816473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6328f-6c89-47e6-8737-cfc20ae7d3ca None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:39.834633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 300/616] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:20:39 2022] PUT /identity/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/76311c7173b442c0af2c745e3e70a07d/roles/091d39715077464daa4cf890db55e6b9 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:20:39.836266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6328f-6c89-47e6-8737-cfc20ae7d3ca None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:39.870649 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 317/617] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:39.912172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e547da8-3d9b-49b3-bf65-3dc42fb0dc93 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:39.943888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:39.962393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:39.994813 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6328f-6c89-47e6-8737-cfc20ae7d3ca None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:40.039523 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762761a9-1b1b-476f-9782-643523574bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:40.040771 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762761a9-1b1b-476f-9782-643523574bef None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:40.041114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762761a9-1b1b-476f-9782-643523574bef None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:40.041454 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762761a9-1b1b-476f-9782-643523574bef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:40.054330 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762761a9-1b1b-476f-9782-643523574bef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:40.056660 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762761a9-1b1b-476f-9782-643523574bef None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:40.079842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-762761a9-1b1b-476f-9782-643523574bef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:40.079842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:40.079842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:40.079842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:40.079842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:40.079842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:40.079842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:40.079842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:40.079842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:40.079842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:40.079842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:40.079842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:40.079842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:40.079842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:40.079842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:40.080874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:40.080874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:40.080874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:40.080874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:40.080874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:40.080874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:40.080874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:40.080874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:40.080874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:40.080874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:40.080874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:40.080874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:40.080874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:40.080874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:40.080874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:40.080874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:40.080874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:40.081796 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:40.081796 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:40.081796 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:40.081796 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:40.084904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 301/618] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:40.090618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762761a9-1b1b-476f-9782-643523574bef None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:40.167113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e547da8-3d9b-49b3-bf65-3dc42fb0dc93 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:40.206017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762761a9-1b1b-476f-9782-643523574bef None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:40.221917 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:40.247967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0d42c-2fed-4369-af94-b72e1aef1815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:40.251043 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0d42c-2fed-4369-af94-b72e1aef1815 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:40.252074 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0d42c-2fed-4369-af94-b72e1aef1815 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:40.254965 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0d42c-2fed-4369-af94-b72e1aef1815 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:40.256544 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0d42c-2fed-4369-af94-b72e1aef1815 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:40.258624 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0d42c-2fed-4369-af94-b72e1aef1815 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:40.262503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0d42c-2fed-4369-af94-b72e1aef1815 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:40.272044 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823a10a-ca4e-4a4f-ac8f-a7e1c4840ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:40.275664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823a10a-ca4e-4a4f-ac8f-a7e1c4840ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:40.284722 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823a10a-ca4e-4a4f-ac8f-a7e1c4840ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:40.285558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823a10a-ca4e-4a4f-ac8f-a7e1c4840ce2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:40.307391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 302/619] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:40.318766 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0d42c-2fed-4369-af94-b72e1aef1815 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:40.320682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823a10a-ca4e-4a4f-ac8f-a7e1c4840ce2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:40.325791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-d823a10a-ca4e-4a4f-ac8f-a7e1c4840ce2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:40.325791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:40.325791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:40.325791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:40.325791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:40.325791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:40.325791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:40.325791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:40.325791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:40.325791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:40.325791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:40.325791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:40.325791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:40.325791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:40.325791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:40.325791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:40.327043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:40.327043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:40.327043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:40.327043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:40.327043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:40.327043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:40.327043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:40.327043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:40.327043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:40.327043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:40.327043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:40.327043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:40.327043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:40.327043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:40.327043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:40.327043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:40.327972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:40.327972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:40.327972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:40.327972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:40.327972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:40.327972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:40.327972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:40.327972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:40.327972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:40.327972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:40.327972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:40.327972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:40.327972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 318/620] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:20:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:40.338905 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823a10a-ca4e-4a4f-ac8f-a7e1c4840ce2 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:40.488120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0d42c-2fed-4369-af94-b72e1aef1815 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:40.508675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:40.530436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999e0da-e73c-4417-84a8-c3c61560742c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:40.531522 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999e0da-e73c-4417-84a8-c3c61560742c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:40.532028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999e0da-e73c-4417-84a8-c3c61560742c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:40.532523 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999e0da-e73c-4417-84a8-c3c61560742c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:40.537023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823a10a-ca4e-4a4f-ac8f-a7e1c4840ce2 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:40.541196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999e0da-e73c-4417-84a8-c3c61560742c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:40.542949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999e0da-e73c-4417-84a8-c3c61560742c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:40.555690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-d999e0da-e73c-4417-84a8-c3c61560742c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:40.555690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:40.555690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:40.555690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:40.555690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:40.555690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:40.555690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:40.555690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:40.555690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:40.555690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:40.555690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:40.555690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:40.555690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:20:40.555690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:20:40.555690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:20:40.556658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:20:40.556658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:40.556658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:40.556658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:40.556658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:40.556658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:40.556658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:40.556658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:40.556658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:40.556658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:40.556658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:40.556658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:40.556658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:40.556658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:40.556658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:40.556658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:40.556658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:40.557565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:20:40.557565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:20:40.557565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:20:40.557565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:20:40.557565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:20:40.557565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:20:40.557565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:40.557565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:40.560157 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 303/621] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:40.561613 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f62e3-306d-4562-9419-c9b2333ae101 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:40.562141 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f62e3-306d-4562-9419-c9b2333ae101 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:40.562614 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f62e3-306d-4562-9419-c9b2333ae101 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:40.568428 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 304/622] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:40.569608 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f62e3-306d-4562-9419-c9b2333ae101 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:40.579792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1caccee-0f45-40c2-b4a7-9c78cb17a294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:40.580431 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caccee-0f45-40c2-b4a7-9c78cb17a294 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:40.580716 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caccee-0f45-40c2-b4a7-9c78cb17a294 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:40.580993 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caccee-0f45-40c2-b4a7-9c78cb17a294 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:40.581783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1caccee-0f45-40c2-b4a7-9c78cb17a294 None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:40.582125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1caccee-0f45-40c2-b4a7-9c78cb17a294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:40.584615 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1caccee-0f45-40c2-b4a7-9c78cb17a294 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:40.600733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 319/623] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:20:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:40.600733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e070bd8-1fd1-4826-aa35-a6772517678a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:40.600733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e070bd8-1fd1-4826-aa35-a6772517678a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:40.600733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e070bd8-1fd1-4826-aa35-a6772517678a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:40.613953 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:40.619706 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-7e070bd8-1fd1-4826-aa35-a6772517678a None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:40.622294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7e070bd8-1fd1-4826-aa35-a6772517678a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:40.694663 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f62e3-306d-4562-9419-c9b2333ae101 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:40.699028 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 320/624] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:40.700168 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e070bd8-1fd1-4826-aa35-a6772517678a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:40.762508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a436a-7948-4302-b198-334bb3d90415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:40.763164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a436a-7948-4302-b198-334bb3d90415 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:40.763516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a436a-7948-4302-b198-334bb3d90415 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:40.763888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a436a-7948-4302-b198-334bb3d90415 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:40.764703 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a436a-7948-4302-b198-334bb3d90415 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:40.765081 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a436a-7948-4302-b198-334bb3d90415 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:40.766639 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a436a-7948-4302-b198-334bb3d90415 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:40.779826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 305/625] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:20:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:40.780269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:40.780798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a436a-7948-4302-b198-334bb3d90415 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:40.828854 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e070bd8-1fd1-4826-aa35-a6772517678a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:40.877188 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316bac9-fce4-404e-8f70-e2ba970271d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:40.878176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316bac9-fce4-404e-8f70-e2ba970271d7 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:40.878671 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316bac9-fce4-404e-8f70-e2ba970271d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:40.879143 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316bac9-fce4-404e-8f70-e2ba970271d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:40.891684 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c316bac9-fce4-404e-8f70-e2ba970271d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:40.893796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c316bac9-fce4-404e-8f70-e2ba970271d7 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:40.902562 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:40.915544 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a436a-7948-4302-b198-334bb3d90415 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:40.918997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-c316bac9-fce4-404e-8f70-e2ba970271d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:40.918997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:40.918997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:40.918997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:40.918997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:40.918997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:40.918997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:40.918997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:40.918997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:40.918997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:40.918997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:40.918997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:40.918997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:40.918997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:40.918997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:40.920186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:40.920186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:40.920186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:40.920186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:40.920186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:40.920186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:40.920186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:40.920186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:40.920186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:40.920186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:40.920186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:40.920186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:40.920186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:40.920186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:40.920186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:40.920186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:40.920186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:40.921406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:40.921406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:40.921406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:40.921406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:40.924370 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 321/626] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:40.926658 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316bac9-fce4-404e-8f70-e2ba970271d7 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:40.979515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f32e21-932e-4ce6-8bd0-448d8ba6c838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:40.980184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f32e21-932e-4ce6-8bd0-448d8ba6c838 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:40.980469 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f32e21-932e-4ce6-8bd0-448d8ba6c838 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:40.980749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f32e21-932e-4ce6-8bd0-448d8ba6c838 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:40.995734 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f32e21-932e-4ce6-8bd0-448d8ba6c838 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:41.003558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-e5f32e21-932e-4ce6-8bd0-448d8ba6c838 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:41.003558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:41.003558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:41.003558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:41.003558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:41.003558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:41.003558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:41.003558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:41.003558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:41.003558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:41.003558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:41.003558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:41.003558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:41.003558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:41.003558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:41.003558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:41.004584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:41.004584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:41.004584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:41.004584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:41.004584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:41.004584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:41.004584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:41.004584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:41.004584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:41.004584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:41.004584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:41.004584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:41.004584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:41.004584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:41.004584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:41.004584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:41.005490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:41.005490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:41.005490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:41.005490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:41.005490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:41.005490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:41.005490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:41.005490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:41.005490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:41.005490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:41.005490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:41.005490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:41.005490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 306/627] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:20:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:41.006306 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f32e21-932e-4ce6-8bd0-448d8ba6c838 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:41.069492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316bac9-fce4-404e-8f70-e2ba970271d7 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:41.092917 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:41.117876 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1605fef-8497-4707-a113-284394526f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:41.119104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1605fef-8497-4707-a113-284394526f9a None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:41.119391 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1605fef-8497-4707-a113-284394526f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:41.119661 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1605fef-8497-4707-a113-284394526f9a None admin] PATH_INFO: `/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/6873a3e440bd4b63b0082fc85beef49c/roles/091d39715077464daa4cf890db55e6b9` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:41.140626 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1605fef-8497-4707-a113-284394526f9a None admin] RBAC: Authorizing `identity:create_grant(project_id=1f4e27cc50bd4f9f8ced4cc544ab3064, user_id=6873a3e440bd4b63b0082fc85beef49c, role_id=091d39715077464daa4cf890db55e6b9)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:41.142869 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1605fef-8497-4707-a113-284394526f9a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:41.148459 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f32e21-932e-4ce6-8bd0-448d8ba6c838 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:41.166483 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 322/628] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:20:40 2022] PUT /identity/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/6873a3e440bd4b63b0082fc85beef49c/roles/091d39715077464daa4cf890db55e6b9 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:20:41.167189 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1605fef-8497-4707-a113-284394526f9a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:41.223593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001e8fb-3e50-457d-82ab-70f6fecf4c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:41.224249 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001e8fb-3e50-457d-82ab-70f6fecf4c20 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:41.224875 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001e8fb-3e50-457d-82ab-70f6fecf4c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:41.225162 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001e8fb-3e50-457d-82ab-70f6fecf4c20 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:41.226831 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001e8fb-3e50-457d-82ab-70f6fecf4c20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:41.227148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001e8fb-3e50-457d-82ab-70f6fecf4c20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:41.232773 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001e8fb-3e50-457d-82ab-70f6fecf4c20 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:41.262060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 307/629] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:41.268174 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001e8fb-3e50-457d-82ab-70f6fecf4c20 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:41.294381 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:41.402698 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1605fef-8497-4707-a113-284394526f9a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:41.440608 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001e8fb-3e50-457d-82ab-70f6fecf4c20 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:41.452175 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830221c5-aa1d-4b1c-a517-39bbf4527bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:41.452882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830221c5-aa1d-4b1c-a517-39bbf4527bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:41.453204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830221c5-aa1d-4b1c-a517-39bbf4527bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:41.453559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830221c5-aa1d-4b1c-a517-39bbf4527bd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:41.455319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830221c5-aa1d-4b1c-a517-39bbf4527bd6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:41.456382 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830221c5-aa1d-4b1c-a517-39bbf4527bd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:41.457725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830221c5-aa1d-4b1c-a517-39bbf4527bd6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:41.469082 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 323/630] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:20:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:41.475882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830221c5-aa1d-4b1c-a517-39bbf4527bd6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:41.499107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f56b364-5e08-41c8-a60c-4490fc954e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:41.500299 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56b364-5e08-41c8-a60c-4490fc954e3e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:41.501297 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56b364-5e08-41c8-a60c-4490fc954e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:41.501743 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56b364-5e08-41c8-a60c-4490fc954e3e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:41.511581 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f56b364-5e08-41c8-a60c-4490fc954e3e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:41.513605 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f56b364-5e08-41c8-a60c-4490fc954e3e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:41.523986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:41.537546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-2f56b364-5e08-41c8-a60c-4490fc954e3e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:20:41.537546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:41.537546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:41.537546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:41.537546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:41.537546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:41.537546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:41.537546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:41.537546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:41.537546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:41.537546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:41.537546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:41.537546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:20:41.537546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:20:41.537546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:20:41.537546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:20:41.538961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:41.538961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:41.538961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:20:41.538961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:41.538961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:20:41.538961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:41.538961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:41.538961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:41.538961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:41.538961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:41.538961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:41.538961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:41.538961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:41.538961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:41.538961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:41.538961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:41.538961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:41.538961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:41.540768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:41.540768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:41.540768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:20:41.540768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:20:41.540768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:20:41.540768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:20:41.540768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:20:41.540768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:20:41.540768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:20:41.540768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:41.543652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 308/631] 10.222.0.83 () {56 vars in 1093 bytes} [Mon Aug 1 15:20:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:20:41.552209 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f56b364-5e08-41c8-a60c-4490fc954e3e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:41.650314 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:41.656275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830221c5-aa1d-4b1c-a517-39bbf4527bd6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:41.691451 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f56b364-5e08-41c8-a60c-4490fc954e3e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:41.733525 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a091bb-c335-4c20-a4b9-d86f1a5bdfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:41.734572 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a091bb-c335-4c20-a4b9-d86f1a5bdfeb None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:41.735081 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a091bb-c335-4c20-a4b9-d86f1a5bdfeb None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:41.735569 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a091bb-c335-4c20-a4b9-d86f1a5bdfeb None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:41.736581 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a091bb-c335-4c20-a4b9-d86f1a5bdfeb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:41.737173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a091bb-c335-4c20-a4b9-d86f1a5bdfeb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:41.738894 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a091bb-c335-4c20-a4b9-d86f1a5bdfeb None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:41.746610 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc0396-19a5-455b-9fb0-57800a121d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:41.747383 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc0396-19a5-455b-9fb0-57800a121d71 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:41.747587 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc0396-19a5-455b-9fb0-57800a121d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:41.747817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc0396-19a5-455b-9fb0-57800a121d71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:41.760407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 309/632] 10.222.0.83 () {56 vars in 1133 bytes} [Mon Aug 1 15:20:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:41.767597 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a091bb-c335-4c20-a4b9-d86f1a5bdfeb None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:41.774354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc0396-19a5-455b-9fb0-57800a121d71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:41.776351 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc0396-19a5-455b-9fb0-57800a121d71 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:41.802840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-27dc0396-19a5-455b-9fb0-57800a121d71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:41.802840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:41.802840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:41.802840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:41.802840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:41.802840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:41.802840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:41.802840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:41.802840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:41.802840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:41.802840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:41.802840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:41.802840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:41.802840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:41.802840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:41.803830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:41.803830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:41.803830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:41.803830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:41.803830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:41.803830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:41.803830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:41.803830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:41.803830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:41.803830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:41.803830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:41.803830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:41.803830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:41.803830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:41.803830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:41.803830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:41.803830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:41.804733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:41.804733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:41.804733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:41.804733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:41.805416 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 324/633] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:41.813149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc0396-19a5-455b-9fb0-57800a121d71 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:41.967190 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a091bb-c335-4c20-a4b9-d86f1a5bdfeb None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:41.968746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:41.990004 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc0396-19a5-455b-9fb0-57800a121d71 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:41.998435 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:42.006404 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3992b97-f20e-41e7-bda5-73afaf243843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:42.007045 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3992b97-f20e-41e7-bda5-73afaf243843 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:42.007330 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3992b97-f20e-41e7-bda5-73afaf243843 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:42.007607 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3992b97-f20e-41e7-bda5-73afaf243843 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:42.036265 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb87ad0-59c7-4385-881d-c400255e5806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:42.037829 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb87ad0-59c7-4385-881d-c400255e5806 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:42.038483 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb87ad0-59c7-4385-881d-c400255e5806 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:42.039300 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb87ad0-59c7-4385-881d-c400255e5806 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:42.041301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb87ad0-59c7-4385-881d-c400255e5806 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:42.042034 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb87ad0-59c7-4385-881d-c400255e5806 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:42.044050 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb87ad0-59c7-4385-881d-c400255e5806 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:42.065849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 325/634] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:42.067712 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b255ad1-0d14-4106-8951-106f5bc49fb4 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:42.068199 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b255ad1-0d14-4106-8951-106f5bc49fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:42.068663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b255ad1-0d14-4106-8951-106f5bc49fb4 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:42.070835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 326/635] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:42.071880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b255ad1-0d14-4106-8951-106f5bc49fb4 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:42.106106 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3992b97-f20e-41e7-bda5-73afaf243843 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:42.107591 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3992b97-f20e-41e7-bda5-73afaf243843 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:42.196002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b255ad1-0d14-4106-8951-106f5bc49fb4 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:42.228401 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 310/636] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:42.228401 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7415a6-c254-4c69-a125-f518978077ba None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:42.228401 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7415a6-c254-4c69-a125-f518978077ba None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:42.228401 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7415a6-c254-4c69-a125-f518978077ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:42.249745 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241aebe-a869-48b5-974d-d8d42e996e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:42.250162 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-8e7415a6-c254-4c69-a125-f518978077ba None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:42.251052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241aebe-a869-48b5-974d-d8d42e996e6c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:42.251481 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8e7415a6-c254-4c69-a125-f518978077ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:42.251738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241aebe-a869-48b5-974d-d8d42e996e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:42.252191 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241aebe-a869-48b5-974d-d8d42e996e6c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:42.261205 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241aebe-a869-48b5-974d-d8d42e996e6c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:42.263033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241aebe-a869-48b5-974d-d8d42e996e6c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:42.278120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:42.280069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-4241aebe-a869-48b5-974d-d8d42e996e6c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 15:20:42.280069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:42.280069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:42.280069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:42.280069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:42.280069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:42.280069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:42.280069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:42.280069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:42.280069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:42.280069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:42.280069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:42.280069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:20:42.280069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:20:42.280069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:20:42.280069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:20:42.281201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:42.281201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:42.281201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:20:42.281201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:42.281201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:20:42.281201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:42.281201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:42.281201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:42.281201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:42.281201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:42.281201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:42.281201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:42.281201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:42.281201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:42.281201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:42.281201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:42.281201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:42.281201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:42.282269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:42.282269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:42.282269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:20:42.282269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:20:42.282269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:20:42.282269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:20:42.282269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:20:42.282269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:20:42.282269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 15:20:42.282269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:42.290236 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 327/637] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:42.291073 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241aebe-a869-48b5-974d-d8d42e996e6c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:42.294753 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:42.331500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 311/638] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:42.332429 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7415a6-c254-4c69-a125-f518978077ba None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:42.423804 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241aebe-a869-48b5-974d-d8d42e996e6c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:42.461520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e6cff-6725-4e67-9fe2-038d3a70a146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:42.462726 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7415a6-c254-4c69-a125-f518978077ba None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:42.462841 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e6cff-6725-4e67-9fe2-038d3a70a146 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:42.463036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e6cff-6725-4e67-9fe2-038d3a70a146 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:42.463296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e6cff-6725-4e67-9fe2-038d3a70a146 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:42.475271 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e6cff-6725-4e67-9fe2-038d3a70a146 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:42.476883 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e6cff-6725-4e67-9fe2-038d3a70a146 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:42.493089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-d96e6cff-6725-4e67-9fe2-038d3a70a146 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:42.493089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:42.493089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:42.493089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:42.493089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:42.493089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:42.493089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:42.493089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:42.493089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:42.493089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:42.493089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:42.493089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:42.493089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:42.493089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:42.493089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:42.494380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:42.494380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:42.494380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:42.494380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:42.494380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:42.494380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:42.494380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:42.494380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:42.494380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:42.494380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:42.494380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:42.494380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:42.494380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:42.494380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:42.494380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:42.494380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:42.494380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:42.495901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:42.495901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:42.495901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:42.495901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:42.495901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 328/639] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:42.495901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2163f54-2e9d-4693-97c8-39d7ecee3e5a None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:42.495901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2163f54-2e9d-4693-97c8-39d7ecee3e5a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:42.496417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2163f54-2e9d-4693-97c8-39d7ecee3e5a None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:42.501813 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 329/640] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:42.502776 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2163f54-2e9d-4693-97c8-39d7ecee3e5a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:42.509699 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f9fde-5b86-4260-be27-d78ae90da70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:42.510472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f9fde-5b86-4260-be27-d78ae90da70c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:42.510745 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f9fde-5b86-4260-be27-d78ae90da70c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:42.511023 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f9fde-5b86-4260-be27-d78ae90da70c None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:42.511831 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f9fde-5b86-4260-be27-d78ae90da70c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:42.512137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f9fde-5b86-4260-be27-d78ae90da70c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:42.513600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f9fde-5b86-4260-be27-d78ae90da70c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:42.542970 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 312/641] 10.222.0.83 () {56 vars in 1137 bytes} [Mon Aug 1 15:20:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:42.548080 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f9fde-5b86-4260-be27-d78ae90da70c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:42.571971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:42.575371 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:42.626955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2163f54-2e9d-4693-97c8-39d7ecee3e5a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:42.669078 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f9fde-5b86-4260-be27-d78ae90da70c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:42.670275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f663e11-23ad-4021-9a63-5aa989fc924e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:42.670964 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f663e11-23ad-4021-9a63-5aa989fc924e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:42.671252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f663e11-23ad-4021-9a63-5aa989fc924e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:42.671524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f663e11-23ad-4021-9a63-5aa989fc924e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:42.683744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f663e11-23ad-4021-9a63-5aa989fc924e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:42.689425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-7f663e11-23ad-4021-9a63-5aa989fc924e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:42.689425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:42.689425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:42.689425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:42.689425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:42.689425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:42.689425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:42.689425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:42.689425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:42.689425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:42.689425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:42.689425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:42.689425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:42.689425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:42.689425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:42.689425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:42.691548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:42.691548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:42.691548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:42.691548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:42.691548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:42.691548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:42.691548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:42.691548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:42.691548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:42.691548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:42.691548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:42.691548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:42.691548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:42.691548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:42.691548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:42.691548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:42.693020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:42.693020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:42.693020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:42.693020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:42.693020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:42.693020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:42.693020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:42.693020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:42.693020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:42.693020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:42.693020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:42.693020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:42.693020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 330/642] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:20:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:42.693020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3ab0b-4a37-4759-b4a3-1059685270fd None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:42.694072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3ab0b-4a37-4759-b4a3-1059685270fd None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:42.694072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3ab0b-4a37-4759-b4a3-1059685270fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:42.712005 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-6ee3ab0b-4a37-4759-b4a3-1059685270fd None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:42.713231 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6ee3ab0b-4a37-4759-b4a3-1059685270fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:42.719976 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac855fba-b64b-472e-aed0-c5f8d349c6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:42.720799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac855fba-b64b-472e-aed0-c5f8d349c6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:42.721420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac855fba-b64b-472e-aed0-c5f8d349c6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:42.721630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac855fba-b64b-472e-aed0-c5f8d349c6b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:42.722361 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac855fba-b64b-472e-aed0-c5f8d349c6b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:42.722702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac855fba-b64b-472e-aed0-c5f8d349c6b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:42.724113 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac855fba-b64b-472e-aed0-c5f8d349c6b3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:42.741480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 313/643] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:42.741480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac855fba-b64b-472e-aed0-c5f8d349c6b3 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:42.798007 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 331/644] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:42.798007 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3ab0b-4a37-4759-b4a3-1059685270fd None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:42.844137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:42.848320 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:42.872740 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac855fba-b64b-472e-aed0-c5f8d349c6b3 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:42.913011 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968cacc-3505-4403-93fe-37844104b387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:42.913640 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968cacc-3505-4403-93fe-37844104b387 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:42.913949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968cacc-3505-4403-93fe-37844104b387 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:42.914056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3ab0b-4a37-4759-b4a3-1059685270fd None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:42.914241 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968cacc-3505-4403-93fe-37844104b387 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:42.955582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0305db6f-1cf1-4724-a8a8-34204bad07f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:42.956171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305db6f-1cf1-4724-a8a8-34204bad07f0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:42.956434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305db6f-1cf1-4724-a8a8-34204bad07f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:42.956751 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305db6f-1cf1-4724-a8a8-34204bad07f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:42.957944 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0305db6f-1cf1-4724-a8a8-34204bad07f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:42.958219 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0305db6f-1cf1-4724-a8a8-34204bad07f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:42.959987 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0305db6f-1cf1-4724-a8a8-34204bad07f0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:42.970472 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 332/645] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:20:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:42.975494 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0305db6f-1cf1-4724-a8a8-34204bad07f0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:42.988626 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968cacc-3505-4403-93fe-37844104b387 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:42.990225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968cacc-3505-4403-93fe-37844104b387 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:43.083148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:43.106555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0305db6f-1cf1-4724-a8a8-34204bad07f0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:43.114031 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 314/646] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:43.115387 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968cacc-3505-4403-93fe-37844104b387 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:43.116472 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:43.147798 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cc241-096a-49fd-b6c8-36917ea7d0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:43.148513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cc241-096a-49fd-b6c8-36917ea7d0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:43.148865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cc241-096a-49fd-b6c8-36917ea7d0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:43.149364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cc241-096a-49fd-b6c8-36917ea7d0f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:43.156955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cc241-096a-49fd-b6c8-36917ea7d0f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:43.158540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cc241-096a-49fd-b6c8-36917ea7d0f8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:43.171192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-273cc241-096a-49fd-b6c8-36917ea7d0f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:43.171192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:43.171192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:43.171192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:43.171192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:43.171192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:43.171192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:43.171192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:43.171192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:43.171192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:43.171192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:43.171192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:43.171192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:20:43.171192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:20:43.171192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:20:43.172154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:20:43.172154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:43.172154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:43.172154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:43.172154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:43.172154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:43.172154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:43.172154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:43.172154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:43.172154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:43.172154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:43.172154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:43.172154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:43.172154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:43.172154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:43.172154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:43.172154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:43.173204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:20:43.173204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:20:43.173204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:20:43.173204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:20:43.173204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:20:43.173204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:20:43.173204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:43.173204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:43.173204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 333/647] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:43.173204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cc241-096a-49fd-b6c8-36917ea7d0f8 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:43.239109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968cacc-3505-4403-93fe-37844104b387 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:43.279584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe0af3-e8ce-45ed-88f7-8aabd5cf9253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:43.280673 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe0af3-e8ce-45ed-88f7-8aabd5cf9253 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:43.280958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe0af3-e8ce-45ed-88f7-8aabd5cf9253 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:43.281228 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe0af3-e8ce-45ed-88f7-8aabd5cf9253 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:43.293411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe0af3-e8ce-45ed-88f7-8aabd5cf9253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:43.295124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cc241-096a-49fd-b6c8-36917ea7d0f8 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:43.298375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-53fe0af3-e8ce-45ed-88f7-8aabd5cf9253 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:43.298375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:43.298375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:43.298375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:43.298375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:43.298375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:43.298375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:43.298375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:43.298375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:43.298375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:43.298375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:43.298375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:43.298375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:43.298375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:43.298375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:43.298375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:43.299480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:43.299480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:43.299480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:43.299480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:43.299480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:43.299480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:43.299480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:43.299480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:43.299480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:43.299480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:43.299480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:43.299480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:43.299480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:43.299480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:43.299480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:43.299480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:43.300490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:43.300490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:43.300490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:43.300490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:43.300490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:43.300490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:43.300490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:43.300490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:43.300490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:43.300490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:43.300490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:43.300490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:43.304526 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 315/648] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:20:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:43.305279 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe0af3-e8ce-45ed-88f7-8aabd5cf9253 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:43.342938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f02e27-5e9e-46bb-8eb6-7511eb39ac67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:43.343706 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f02e27-5e9e-46bb-8eb6-7511eb39ac67 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:43.344008 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f02e27-5e9e-46bb-8eb6-7511eb39ac67 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:43.344289 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f02e27-5e9e-46bb-8eb6-7511eb39ac67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:43.350162 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:43.358308 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f02e27-5e9e-46bb-8eb6-7511eb39ac67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:43.359779 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f02e27-5e9e-46bb-8eb6-7511eb39ac67 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:43.375260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-a8f02e27-5e9e-46bb-8eb6-7511eb39ac67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:43.375260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:43.375260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:43.375260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:43.375260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:43.375260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:43.375260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:43.375260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:43.375260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:43.375260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:43.375260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:43.375260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:43.375260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:43.375260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:43.375260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:43.376443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:43.376443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:43.376443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:43.376443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:43.376443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:43.376443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:43.376443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:43.376443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:43.376443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:43.376443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:43.376443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:43.376443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:43.376443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:43.376443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:43.376443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:43.376443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:43.376443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:43.377698 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:43.377698 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:43.377698 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:43.377698 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:43.377698 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 334/649] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:43.377698 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f02e27-5e9e-46bb-8eb6-7511eb39ac67 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:43.402505 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:43.432498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe0af3-e8ce-45ed-88f7-8aabd5cf9253 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:43.474091 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3504a0b-2e21-4283-b589-eb05d3630541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:43.475483 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3504a0b-2e21-4283-b589-eb05d3630541 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:43.475924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3504a0b-2e21-4283-b589-eb05d3630541 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:43.476263 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3504a0b-2e21-4283-b589-eb05d3630541 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:43.487448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3504a0b-2e21-4283-b589-eb05d3630541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:43.489466 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3504a0b-2e21-4283-b589-eb05d3630541 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:43.506063 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f02e27-5e9e-46bb-8eb6-7511eb39ac67 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:43.506827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-d3504a0b-2e21-4283-b589-eb05d3630541 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:43.506827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:43.506827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:43.506827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:43.506827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:43.506827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:43.506827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:43.506827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:43.506827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:43.506827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:43.506827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:43.506827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:43.506827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:43.506827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:43.506827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:43.507863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:43.507863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:43.507863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:43.507863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:43.507863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:43.507863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:43.507863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:43.507863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:43.507863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:43.507863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:43.507863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:43.507863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:43.507863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:43.507863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:43.507863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:43.507863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:43.507863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:43.508900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:43.508900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:43.508900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:43.508900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:43.508900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 316/650] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:43.508900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3504a0b-2e21-4283-b589-eb05d3630541 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:43.543757 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb110868-acc5-4254-a41c-05efa0c578a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:43.544373 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb110868-acc5-4254-a41c-05efa0c578a9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:43.544657 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb110868-acc5-4254-a41c-05efa0c578a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:43.544927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb110868-acc5-4254-a41c-05efa0c578a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:43.545677 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb110868-acc5-4254-a41c-05efa0c578a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:43.546013 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb110868-acc5-4254-a41c-05efa0c578a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:43.547446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb110868-acc5-4254-a41c-05efa0c578a9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:43.559702 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 335/651] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:20:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:43.560415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb110868-acc5-4254-a41c-05efa0c578a9 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:43.608891 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:43.619464 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3504a0b-2e21-4283-b589-eb05d3630541 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:43.645213 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:43.657758 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bf28d-f6cf-4cce-920d-64bd0c28293d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:43.658541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bf28d-f6cf-4cce-920d-64bd0c28293d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:43.658846 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bf28d-f6cf-4cce-920d-64bd0c28293d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:43.659234 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bf28d-f6cf-4cce-920d-64bd0c28293d None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:43.660071 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bf28d-f6cf-4cce-920d-64bd0c28293d None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:43.660482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bf28d-f6cf-4cce-920d-64bd0c28293d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:43.662203 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bf28d-f6cf-4cce-920d-64bd0c28293d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:43.668047 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb110868-acc5-4254-a41c-05efa0c578a9 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:43.672576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 317/652] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:20:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:43.673958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4601dd-3a2f-4470-a197-382181757fa8 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:43.674353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4601dd-3a2f-4470-a197-382181757fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:43.674735 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4601dd-3a2f-4470-a197-382181757fa8 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:43.676302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 318/653] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:43.677122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4601dd-3a2f-4470-a197-382181757fa8 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:43.708144 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9732d0-95b5-4453-9d9e-764dbfc73459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:43.708766 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9732d0-95b5-4453-9d9e-764dbfc73459 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:43.709051 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9732d0-95b5-4453-9d9e-764dbfc73459 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:43.709334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9732d0-95b5-4453-9d9e-764dbfc73459 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:43.710054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9732d0-95b5-4453-9d9e-764dbfc73459 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:43.710369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9732d0-95b5-4453-9d9e-764dbfc73459 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:43.711725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9732d0-95b5-4453-9d9e-764dbfc73459 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:43.724017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 336/654] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:43.724914 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9732d0-95b5-4453-9d9e-764dbfc73459 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:43.796493 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4601dd-3a2f-4470-a197-382181757fa8 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:43.835155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77cb90e-6e7a-44f3-a120-ade0f404bb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:43.835687 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77cb90e-6e7a-44f3-a120-ade0f404bb5a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:43.835981 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77cb90e-6e7a-44f3-a120-ade0f404bb5a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:43.836251 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77cb90e-6e7a-44f3-a120-ade0f404bb5a None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:43.836351 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9732d0-95b5-4453-9d9e-764dbfc73459 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:43.837500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77cb90e-6e7a-44f3-a120-ade0f404bb5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:43.837844 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77cb90e-6e7a-44f3-a120-ade0f404bb5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:43.839231 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77cb90e-6e7a-44f3-a120-ade0f404bb5a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:43.851318 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 319/655] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:43.852149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77cb90e-6e7a-44f3-a120-ade0f404bb5a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:43.866904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:43.874452 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fab33-3740-4ac5-975e-a5340c7662a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:43.875077 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fab33-3740-4ac5-975e-a5340c7662a8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:43.875359 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fab33-3740-4ac5-975e-a5340c7662a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:43.875635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fab33-3740-4ac5-975e-a5340c7662a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:43.876422 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fab33-3740-4ac5-975e-a5340c7662a8 None admin] RBAC: Adding query filter params (role.id=091d39715077464daa4cf890db55e6b9, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=25dc41ffc41c47fdbeec06110d04610c) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:43.876731 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fab33-3740-4ac5-975e-a5340c7662a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:43.878197 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fab33-3740-4ac5-975e-a5340c7662a8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:43.884804 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 337/656] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:20:43 2022] GET /identity/v3/role_assignments?user.id=25dc41ffc41c47fdbeec06110d04610c&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=091d39715077464daa4cf890db55e6b9 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:43.886032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b23c8-b3c6-4270-a878-5be7d62d46c3 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:43.886329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b23c8-b3c6-4270-a878-5be7d62d46c3 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:43.887016 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b23c8-b3c6-4270-a878-5be7d62d46c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:43.893448 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:43.915474 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-eb9b23c8-b3c6-4270-a878-5be7d62d46c3 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:43.916737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-eb9b23c8-b3c6-4270-a878-5be7d62d46c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:43.984474 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77cb90e-6e7a-44f3-a120-ade0f404bb5a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:44.018090 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 338/657] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:44.019348 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b23c8-b3c6-4270-a878-5be7d62d46c3 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:44.034497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dceb3e-86bf-4879-b620-aa7132d97e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:44.034766 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dceb3e-86bf-4879-b620-aa7132d97e4b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:44.034887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dceb3e-86bf-4879-b620-aa7132d97e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:44.035129 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dceb3e-86bf-4879-b620-aa7132d97e4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:44.047307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dceb3e-86bf-4879-b620-aa7132d97e4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:44.049035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dceb3e-86bf-4879-b620-aa7132d97e4b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:44.067542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-c2dceb3e-86bf-4879-b620-aa7132d97e4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:44.067542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:44.067542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:44.067542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:44.067542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:44.067542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:44.067542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:44.067542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:44.067542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:44.067542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:44.067542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:44.067542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:44.067542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:44.067542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:44.067542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:44.068492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:44.068492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:44.068492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:44.068492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:44.068492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:44.068492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:44.068492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:44.068492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:44.068492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:44.068492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:44.068492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:44.068492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:44.068492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:44.068492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:44.068492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:44.068492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:44.068492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:44.069818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:44.069818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:44.069818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:44.069818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:44.069818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 320/658] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:44.069818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dceb3e-86bf-4879-b620-aa7132d97e4b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:44.158724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:44.163228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b23c8-b3c6-4270-a878-5be7d62d46c3 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:44.198823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:44.203244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dceb3e-86bf-4879-b620-aa7132d97e4b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:44.205301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02fce7-073c-47e7-a15b-9a4363833197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:44.206472 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02fce7-073c-47e7-a15b-9a4363833197 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:44.206472 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02fce7-073c-47e7-a15b-9a4363833197 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:44.206674 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02fce7-073c-47e7-a15b-9a4363833197 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:44.215298 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea02fce7-073c-47e7-a15b-9a4363833197 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:44.217033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea02fce7-073c-47e7-a15b-9a4363833197 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:44.234896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-ea02fce7-073c-47e7-a15b-9a4363833197 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:20:44.234896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:44.234896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:44.234896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:44.234896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:44.234896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:44.234896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:44.234896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:44.234896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:44.234896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:44.234896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:44.234896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:44.234896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:20:44.234896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:20:44.234896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:20:44.234896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:20:44.236154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:44.236154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:44.236154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:20:44.236154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:44.236154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:20:44.236154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:44.236154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:44.236154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:44.236154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:44.236154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:44.236154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:44.236154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:44.236154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:44.236154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:44.236154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:44.236154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:44.236154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:44.236154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:44.237855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:44.237855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:44.237855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:20:44.237855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:20:44.237855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:20:44.237855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:20:44.237855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:20:44.237855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:20:44.237855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:20:44.237855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:44.237855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 339/659] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:44.237855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02fce7-073c-47e7-a15b-9a4363833197 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:44.244763 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077dfdf-d988-4a30-a772-4f700b73a8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:44.245432 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077dfdf-d988-4a30-a772-4f700b73a8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:44.245683 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077dfdf-d988-4a30-a772-4f700b73a8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:44.245982 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077dfdf-d988-4a30-a772-4f700b73a8b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:44.253952 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7077dfdf-d988-4a30-a772-4f700b73a8b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:44.255625 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7077dfdf-d988-4a30-a772-4f700b73a8b4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:44.269645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-7077dfdf-d988-4a30-a772-4f700b73a8b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:44.269645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:44.269645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:44.269645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:44.269645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:44.269645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:44.269645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:44.269645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:44.269645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:44.269645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:44.269645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:44.269645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:44.269645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:20:44.269645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:20:44.269645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:20:44.270674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:20:44.270674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:44.270674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:44.270674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:44.270674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:44.270674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:44.270674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:44.270674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:44.270674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:44.270674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:44.270674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:44.270674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:44.270674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:44.270674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:44.270674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:44.270674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:44.270674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:44.271725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:20:44.271725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:20:44.271725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:20:44.271725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:20:44.271725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:20:44.271725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:20:44.271725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:44.271725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:44.271725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 321/660] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:44.271725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077dfdf-d988-4a30-a772-4f700b73a8b4 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:44.367388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02fce7-073c-47e7-a15b-9a4363833197 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:44.387395 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077dfdf-d988-4a30-a772-4f700b73a8b4 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:44.406376 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ebd99-03c8-48f1-9497-45b5358b72ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:44.407028 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ebd99-03c8-48f1-9497-45b5358b72ca None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:44.407307 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ebd99-03c8-48f1-9497-45b5358b72ca None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:44.407583 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ebd99-03c8-48f1-9497-45b5358b72ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:44.415713 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:44.420387 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ebd99-03c8-48f1-9497-45b5358b72ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:44.425866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-179ebd99-03c8-48f1-9497-45b5358b72ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:44.425866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:44.425866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:44.425866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:44.425866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:44.425866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:44.425866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:44.425866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:44.425866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:44.425866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:44.425866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:44.425866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:44.425866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:44.425866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:44.425866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:44.425866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:44.427114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:44.427114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:44.427114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:44.427114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:44.427114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:44.427114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:44.427114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:44.427114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:44.427114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:44.427114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:44.427114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:44.427114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:44.427114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:44.427114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:44.427114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:44.427114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:44.428079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:44.428079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:44.428079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:44.428079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:44.428079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:44.428079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:44.428079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:44.428079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:44.428079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:44.428079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:44.428079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:44.428079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:44.428079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 340/661] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:20:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:44.433687 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ebd99-03c8-48f1-9497-45b5358b72ca None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:44.435260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0971e26-9127-41e5-83ef-a16914ff472b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:44.435912 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0971e26-9127-41e5-83ef-a16914ff472b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:44.436225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0971e26-9127-41e5-83ef-a16914ff472b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:44.436505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0971e26-9127-41e5-83ef-a16914ff472b None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:44.437214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0971e26-9127-41e5-83ef-a16914ff472b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:44.437523 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0971e26-9127-41e5-83ef-a16914ff472b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:44.439034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0971e26-9127-41e5-83ef-a16914ff472b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:44.464010 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 322/662] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:44.464010 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0971e26-9127-41e5-83ef-a16914ff472b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:44.510767 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:44.568417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ebd99-03c8-48f1-9497-45b5358b72ca None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:44.587978 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0971e26-9127-41e5-83ef-a16914ff472b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:44.611088 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a16447-9376-46a7-ab4b-9fa3f50c1713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:44.612024 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a16447-9376-46a7-ab4b-9fa3f50c1713 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:44.612507 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a16447-9376-46a7-ab4b-9fa3f50c1713 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:44.612789 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a16447-9376-46a7-ab4b-9fa3f50c1713 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:44.613592 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a16447-9376-46a7-ab4b-9fa3f50c1713 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:44.613926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a16447-9376-46a7-ab4b-9fa3f50c1713 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:44.615437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a16447-9376-46a7-ab4b-9fa3f50c1713 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:44.632956 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a74da-7cc2-417b-b14d-bd00c4e9746d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:44.633599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a74da-7cc2-417b-b14d-bd00c4e9746d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:44.633999 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a74da-7cc2-417b-b14d-bd00c4e9746d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:44.634212 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a74da-7cc2-417b-b14d-bd00c4e9746d None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:44.634970 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a74da-7cc2-417b-b14d-bd00c4e9746d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:44.635299 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a74da-7cc2-417b-b14d-bd00c4e9746d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:44.636733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a74da-7cc2-417b-b14d-bd00c4e9746d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:44.645992 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 341/663] 10.222.0.83 () {56 vars in 1137 bytes} [Mon Aug 1 15:20:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:44.645992 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a16447-9376-46a7-ab4b-9fa3f50c1713 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:44.654007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 323/664] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:20:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:44.654007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a74da-7cc2-417b-b14d-bd00c4e9746d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:44.717031 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:44.776171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a16447-9376-46a7-ab4b-9fa3f50c1713 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:44.780689 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a74da-7cc2-417b-b14d-bd00c4e9746d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:44.795084 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:44.820449 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45251d5-a832-4597-b396-b5b5b78db1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:44.821235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45251d5-a832-4597-b396-b5b5b78db1eb None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:44.821593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45251d5-a832-4597-b396-b5b5b78db1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:44.821943 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45251d5-a832-4597-b396-b5b5b78db1eb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:44.826717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75ea9b-1f08-46b9-b5b0-76b17de3ed46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:44.827530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75ea9b-1f08-46b9-b5b0-76b17de3ed46 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:44.827910 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75ea9b-1f08-46b9-b5b0-76b17de3ed46 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:44.828303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75ea9b-1f08-46b9-b5b0-76b17de3ed46 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:44.829280 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75ea9b-1f08-46b9-b5b0-76b17de3ed46 None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:44.829612 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75ea9b-1f08-46b9-b5b0-76b17de3ed46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:44.830239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45251d5-a832-4597-b396-b5b5b78db1eb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:44.831490 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75ea9b-1f08-46b9-b5b0-76b17de3ed46 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:44.831689 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45251d5-a832-4597-b396-b5b5b78db1eb None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:44.847995 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 342/665] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:20:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:44.847995 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75ea9b-1f08-46b9-b5b0-76b17de3ed46 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:44.848415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-a45251d5-a832-4597-b396-b5b5b78db1eb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 15:20:44.848415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:44.848415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:44.848415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:44.848415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:44.848415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:44.848415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:44.848415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:44.848415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:44.848415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:44.848415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:44.848415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:44.848415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:20:44.848415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:20:44.848415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:20:44.848415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:20:44.849824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:44.849824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:44.849824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:20:44.849824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:44.849824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:20:44.849824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:44.849824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:44.849824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:44.849824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:44.849824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:44.849824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:44.849824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:44.849824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:44.849824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:44.849824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:44.849824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:44.849824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:44.849824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:44.851935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:44.851935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:44.851935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:20:44.851935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:20:44.851935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:20:44.851935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:20:44.851935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:20:44.851935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:20:44.851935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 15:20:44.851935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:44.851935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 324/666] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:20:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:44.851935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45251d5-a832-4597-b396-b5b5b78db1eb None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:44.980321 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45251d5-a832-4597-b396-b5b5b78db1eb None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:44.982889 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:44.985601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75ea9b-1f08-46b9-b5b0-76b17de3ed46 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:45.019020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf360ab-b2b6-42d3-8ccb-1ca834e49c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:45.020043 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf360ab-b2b6-42d3-8ccb-1ca834e49c68 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:45.020715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf360ab-b2b6-42d3-8ccb-1ca834e49c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:45.021163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf360ab-b2b6-42d3-8ccb-1ca834e49c68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.022120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf360ab-b2b6-42d3-8ccb-1ca834e49c68 None admin] RBAC: Adding query filter params (role.id=091d39715077464daa4cf890db55e6b9, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=69d216a568b546589e854c2b4476efe6) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:45.022472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf360ab-b2b6-42d3-8ccb-1ca834e49c68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:45.024176 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf360ab-b2b6-42d3-8ccb-1ca834e49c68 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:45.025631 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed64e1-7686-4135-ab8c-de556a935478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:45.026470 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed64e1-7686-4135-ab8c-de556a935478 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:45.026892 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed64e1-7686-4135-ab8c-de556a935478 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:45.027310 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed64e1-7686-4135-ab8c-de556a935478 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.041920 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 325/667] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:20:44 2022] GET /identity/v3/role_assignments?user.id=69d216a568b546589e854c2b4476efe6&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=091d39715077464daa4cf890db55e6b9 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:45.041920 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf360ab-b2b6-42d3-8ccb-1ca834e49c68 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:45.098444 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:45.148270 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed64e1-7686-4135-ab8c-de556a935478 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:45.149966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed64e1-7686-4135-ab8c-de556a935478 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:45.175522 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf360ab-b2b6-42d3-8ccb-1ca834e49c68 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:45.225456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6414fc52-e82a-4b20-ae11-0bda1644b635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:45.226517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414fc52-e82a-4b20-ae11-0bda1644b635 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:45.226809 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414fc52-e82a-4b20-ae11-0bda1644b635 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:45.227395 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414fc52-e82a-4b20-ae11-0bda1644b635 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.228809 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6414fc52-e82a-4b20-ae11-0bda1644b635 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:45.229125 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6414fc52-e82a-4b20-ae11-0bda1644b635 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:45.231351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6414fc52-e82a-4b20-ae11-0bda1644b635 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:45.266017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 326/668] 10.222.0.83 () {56 vars in 1139 bytes} [Mon Aug 1 15:20:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:45.267603 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6414fc52-e82a-4b20-ae11-0bda1644b635 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:45.315176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 343/669] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:45.316002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed64e1-7686-4135-ab8c-de556a935478 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:45.321623 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:45.407019 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:45.423256 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6414fc52-e82a-4b20-ae11-0bda1644b635 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:45.444878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed64e1-7686-4135-ab8c-de556a935478 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:45.469051 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fe9b8-2d9e-44a0-989b-bb082297eb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:45.469746 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fe9b8-2d9e-44a0-989b-bb082297eb82 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:45.470595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fe9b8-2d9e-44a0-989b-bb082297eb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:45.470884 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fe9b8-2d9e-44a0-989b-bb082297eb82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.485763 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fe9b8-2d9e-44a0-989b-bb082297eb82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:45.487562 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fe9b8-2d9e-44a0-989b-bb082297eb82 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:45.488040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855368c-e00f-4a0f-8bfa-c9191d834466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:45.488857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855368c-e00f-4a0f-8bfa-c9191d834466 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:45.489299 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855368c-e00f-4a0f-8bfa-c9191d834466 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:45.489915 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855368c-e00f-4a0f-8bfa-c9191d834466 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.507752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-9c4fe9b8-2d9e-44a0-989b-bb082297eb82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:45.507752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:45.507752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:45.507752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:45.507752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:45.507752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:45.507752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:45.507752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:45.507752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:45.507752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:45.507752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:45.507752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:45.507752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:45.507752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:45.507752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:45.508832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:45.508832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:45.508832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:45.508832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:45.508832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:45.508832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:45.508832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:45.508832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:45.508832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:45.508832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:45.508832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:45.508832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:45.508832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:45.508832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:45.508832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:45.508832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:45.508832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:45.510531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:45.510531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:45.510531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:45.510531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:45.510531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 327/670] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:45.510872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fe9b8-2d9e-44a0-989b-bb082297eb82 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:45.591691 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855368c-e00f-4a0f-8bfa-c9191d834466 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:45.593482 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855368c-e00f-4a0f-8bfa-c9191d834466 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:45.629684 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:45.665581 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fe9b8-2d9e-44a0-989b-bb082297eb82 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:45.714873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6493afb-9da4-4ed8-9a1f-0c0eefca4c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:45.716239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6493afb-9da4-4ed8-9a1f-0c0eefca4c86 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:45.716626 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6493afb-9da4-4ed8-9a1f-0c0eefca4c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:45.717025 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6493afb-9da4-4ed8-9a1f-0c0eefca4c86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.720762 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:45.733212 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6493afb-9da4-4ed8-9a1f-0c0eefca4c86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:45.735413 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6493afb-9da4-4ed8-9a1f-0c0eefca4c86 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:45.752969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-e6493afb-9da4-4ed8-9a1f-0c0eefca4c86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:45.752969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:45.752969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:45.752969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:45.752969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:45.752969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:45.752969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:45.752969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:45.752969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:45.752969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:45.752969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:45.752969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:45.752969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:45.752969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:45.752969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:45.754158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:45.754158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:45.754158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:45.754158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:45.754158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:45.754158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:45.754158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:45.754158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:45.754158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:45.754158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:45.754158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:45.754158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:45.754158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:45.754158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:45.754158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:45.754158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:45.754158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:45.755268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:45.755268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:45.755268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:45.755268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:45.756376 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 344/671] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:45.756637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 328/672] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:45.759270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6493afb-9da4-4ed8-9a1f-0c0eefca4c86 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:45.759412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855368c-e00f-4a0f-8bfa-c9191d834466 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:45.879342 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855368c-e00f-4a0f-8bfa-c9191d834466 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:45.895764 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6493afb-9da4-4ed8-9a1f-0c0eefca4c86 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:45.920948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54c806-39e0-4c5e-a8c7-f119484d4d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:45.921827 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54c806-39e0-4c5e-a8c7-f119484d4d04 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:45.922450 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54c806-39e0-4c5e-a8c7-f119484d4d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:45.923031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54c806-39e0-4c5e-a8c7-f119484d4d04 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.923922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54c806-39e0-4c5e-a8c7-f119484d4d04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:45.924362 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54c806-39e0-4c5e-a8c7-f119484d4d04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:45.926170 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54c806-39e0-4c5e-a8c7-f119484d4d04 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:45.932946 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:45.941147 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f8208-3af7-4e76-b751-135922963b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:45.942103 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f8208-3af7-4e76-b751-135922963b6f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:45.942557 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f8208-3af7-4e76-b751-135922963b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:45.943017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f8208-3af7-4e76-b751-135922963b6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.947350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 345/673] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:45.948195 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54c806-39e0-4c5e-a8c7-f119484d4d04 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:45.957762 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f8208-3af7-4e76-b751-135922963b6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:45.960011 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f8208-3af7-4e76-b751-135922963b6f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:45.981149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-496f8208-3af7-4e76-b751-135922963b6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:45.981149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:45.981149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:45.981149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:45.981149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:45.981149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:45.981149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:45.981149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:45.981149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:45.981149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:45.981149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:45.981149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:45.981149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:45.981149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:45.981149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:45.982370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:45.982370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:45.982370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:45.982370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:45.982370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:45.982370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:45.982370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:45.982370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:45.982370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:45.982370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:45.982370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:45.982370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:45.982370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:45.982370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:45.982370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:45.982370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:45.982370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:45.983462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:45.983462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:45.983462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:45.983462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:45.987061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:45.990435 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 329/674] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:45.991524 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f8208-3af7-4e76-b751-135922963b6f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:46.096412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54c806-39e0-4c5e-a8c7-f119484d4d04 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:46.120526 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f8208-3af7-4e76-b751-135922963b6f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:46.148701 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd1d15-b784-41d8-81d5-972e989550a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:46.150208 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd1d15-b784-41d8-81d5-972e989550a4 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:46.150505 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd1d15-b784-41d8-81d5-972e989550a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:46.151192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd1d15-b784-41d8-81d5-972e989550a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:46.152184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd1d15-b784-41d8-81d5-972e989550a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:46.152496 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd1d15-b784-41d8-81d5-972e989550a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:46.155075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd1d15-b784-41d8-81d5-972e989550a4 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:46.160294 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8644b-89cf-492c-827f-f16a34bb3118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:46.161705 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8644b-89cf-492c-827f-f16a34bb3118 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:46.161705 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8644b-89cf-492c-827f-f16a34bb3118 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:46.161705 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8644b-89cf-492c-827f-f16a34bb3118 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:46.172763 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 346/675] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:46.172967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8644b-89cf-492c-827f-f16a34bb3118 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:46.173054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd1d15-b784-41d8-81d5-972e989550a4 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:46.173143 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8644b-89cf-492c-827f-f16a34bb3118 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:46.184161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-29d8644b-89cf-492c-827f-f16a34bb3118 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:20:46.184161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:46.184161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:46.184161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:46.184161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:46.184161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:46.184161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:46.184161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:46.184161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:46.184161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:46.184161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:46.184161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:46.184161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:20:46.184161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:20:46.184161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:20:46.184161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:20:46.185191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:46.185191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:46.185191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:20:46.185191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:46.185191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:20:46.185191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:46.185191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:46.185191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:46.185191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:46.185191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:46.185191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:46.185191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:46.185191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:46.185191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:46.185191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:46.185191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:46.185191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:46.185191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:46.186177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:46.186177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:46.186177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:20:46.186177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:20:46.186177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:20:46.186177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:20:46.186177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:20:46.186177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:20:46.186177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:20:46.186177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:46.186177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 330/676] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:20:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:46.186814 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4d5e0-e42c-438b-bb5e-739df66b5f8c None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:46.187466 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4d5e0-e42c-438b-bb5e-739df66b5f8c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:46.187466 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4d5e0-e42c-438b-bb5e-739df66b5f8c None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:46.189007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 331/677] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:46.189940 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4d5e0-e42c-438b-bb5e-739df66b5f8c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:46.240212 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:46.288227 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:46.300043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd1d15-b784-41d8-81d5-972e989550a4 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:46.302163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4d5e0-e42c-438b-bb5e-739df66b5f8c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:46.339704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c0471-a0e8-443c-aad0-814eb1066e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:46.340086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6326f83-3dec-4467-845a-d040af42ddc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:46.340280 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c0471-a0e8-443c-aad0-814eb1066e24 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:46.340595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c0471-a0e8-443c-aad0-814eb1066e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:46.340872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c0471-a0e8-443c-aad0-814eb1066e24 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:46.340971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6326f83-3dec-4467-845a-d040af42ddc7 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:46.341326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6326f83-3dec-4467-845a-d040af42ddc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:46.341577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c0471-a0e8-443c-aad0-814eb1066e24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:46.341685 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6326f83-3dec-4467-845a-d040af42ddc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:46.341895 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c0471-a0e8-443c-aad0-814eb1066e24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:46.343391 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c0471-a0e8-443c-aad0-814eb1066e24 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:46.350943 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6326f83-3dec-4467-845a-d040af42ddc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:46.352641 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6326f83-3dec-4467-845a-d040af42ddc7 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:46.357316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 347/678] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:46.358470 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c0471-a0e8-443c-aad0-814eb1066e24 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:46.367123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-d6326f83-3dec-4467-845a-d040af42ddc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:46.367123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:46.367123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:46.367123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:46.367123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:46.367123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:46.367123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:46.367123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:46.367123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:46.367123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:46.367123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:46.367123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:46.367123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:20:46.367123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:20:46.367123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:20:46.368600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:20:46.368600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:46.368600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:46.368600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:46.368600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:46.368600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:46.368600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:46.368600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:46.368600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:46.368600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:46.368600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:46.368600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:46.368600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:46.368600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:46.368600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:46.368600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:46.368600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:46.370164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:20:46.370164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:20:46.370164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:20:46.370164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:20:46.370164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:20:46.370164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:20:46.370164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:46.370164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:46.370164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 332/679] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:46.370675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907b751-c533-4706-957d-79bd21cfed36 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:46.370675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907b751-c533-4706-957d-79bd21cfed36 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:46.371030 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907b751-c533-4706-957d-79bd21cfed36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:46.389529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c907b751-c533-4706-957d-79bd21cfed36 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:46.390780 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c907b751-c533-4706-957d-79bd21cfed36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:46.482128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 333/680] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:46.482128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907b751-c533-4706-957d-79bd21cfed36 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:46.516917 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c0471-a0e8-443c-aad0-814eb1066e24 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:46.522466 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:46.565782 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae1fe4-baf8-4796-9972-28dc429a0803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:46.566513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae1fe4-baf8-4796-9972-28dc429a0803 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:46.566952 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae1fe4-baf8-4796-9972-28dc429a0803 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:46.567271 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae1fe4-baf8-4796-9972-28dc429a0803 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:46.568199 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae1fe4-baf8-4796-9972-28dc429a0803 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:46.568514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae1fe4-baf8-4796-9972-28dc429a0803 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:46.569989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae1fe4-baf8-4796-9972-28dc429a0803 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:46.590516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 348/681] 10.222.0.83 () {56 vars in 1135 bytes} [Mon Aug 1 15:20:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:46.595615 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae1fe4-baf8-4796-9972-28dc429a0803 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:46.610288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:46.625544 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907b751-c533-4706-957d-79bd21cfed36 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:46.690591 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07453e59-aae4-47ed-ae6b-75b50fa9d116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:46.691442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07453e59-aae4-47ed-ae6b-75b50fa9d116 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:46.691721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07453e59-aae4-47ed-ae6b-75b50fa9d116 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:46.691989 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07453e59-aae4-47ed-ae6b-75b50fa9d116 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:46.699862 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07453e59-aae4-47ed-ae6b-75b50fa9d116 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:46.701413 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07453e59-aae4-47ed-ae6b-75b50fa9d116 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:46.722497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-07453e59-aae4-47ed-ae6b-75b50fa9d116 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:46.722497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:46.722497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:46.722497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:46.722497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:46.722497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:46.722497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:46.722497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:46.722497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:46.722497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:46.722497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:46.722497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:46.722497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:20:46.722497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:20:46.722497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:20:46.724200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:20:46.724200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:46.724200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:46.724200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:46.724200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:46.724200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:46.724200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:46.724200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:46.724200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:46.724200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:46.724200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:46.724200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:46.724200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:46.724200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:46.724200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:46.724200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:46.724200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:46.727102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:20:46.727102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:20:46.727102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:20:46.727102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:20:46.727102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:20:46.727102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:20:46.727102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:46.727102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:46.727102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 334/682] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:46.727102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07453e59-aae4-47ed-ae6b-75b50fa9d116 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:46.736716 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae1fe4-baf8-4796-9972-28dc429a0803 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:46.793605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c1b42-127c-4661-b62f-22c8a497227d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:46.794428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c1b42-127c-4661-b62f-22c8a497227d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:46.794811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c1b42-127c-4661-b62f-22c8a497227d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:46.795185 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c1b42-127c-4661-b62f-22c8a497227d None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:46.796050 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c1b42-127c-4661-b62f-22c8a497227d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:46.796462 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c1b42-127c-4661-b62f-22c8a497227d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:46.798025 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c1b42-127c-4661-b62f-22c8a497227d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:46.812168 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 349/683] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:20:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:46.813171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c1b42-127c-4661-b62f-22c8a497227d None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:46.859867 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:46.872241 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07453e59-aae4-47ed-ae6b-75b50fa9d116 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:46.914112 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4bd20-488a-4236-abfc-267f9e52122e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:46.915525 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4bd20-488a-4236-abfc-267f9e52122e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:46.915926 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4bd20-488a-4236-abfc-267f9e52122e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:46.916353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4bd20-488a-4236-abfc-267f9e52122e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:46.924981 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:46.928706 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4bd20-488a-4236-abfc-267f9e52122e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:46.934669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-5fb4bd20-488a-4236-abfc-267f9e52122e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:20:46.934669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:46.934669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:46.934669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:46.934669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:46.934669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:46.934669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:46.934669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:46.934669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:46.934669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:46.934669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:46.934669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:46.934669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:46.934669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:46.934669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:46.934669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:46.935990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:46.935990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:46.935990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:46.935990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:46.935990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:46.935990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:46.935990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:46.935990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:46.935990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:46.935990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:46.935990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:46.935990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:46.935990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:46.935990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:46.935990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:46.935990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:46.937035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:46.937035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:46.937035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:46.937035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:46.937035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:46.937035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:46.937035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:46.937035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:46.937035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:46.937035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:46.937035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:20:46.937035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:46.945702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 335/684] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:20:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:46.945702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4bd20-488a-4236-abfc-267f9e52122e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:46.974397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c1b42-127c-4661-b62f-22c8a497227d None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:47.030639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813872cc-f5c1-4678-b236-95edf65eef4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:47.032264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813872cc-f5c1-4678-b236-95edf65eef4f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:47.032264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813872cc-f5c1-4678-b236-95edf65eef4f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:47.032475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813872cc-f5c1-4678-b236-95edf65eef4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:47.103906 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4bd20-488a-4236-abfc-267f9e52122e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:47.144800 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813872cc-f5c1-4678-b236-95edf65eef4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:47.146464 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813872cc-f5c1-4678-b236-95edf65eef4f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:47.146830 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50af850-f4ff-4d5f-b6e6-c1a193cc9d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:47.147777 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50af850-f4ff-4d5f-b6e6-c1a193cc9d10 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:47.148259 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50af850-f4ff-4d5f-b6e6-c1a193cc9d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:47.148797 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50af850-f4ff-4d5f-b6e6-c1a193cc9d10 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:47.150450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50af850-f4ff-4d5f-b6e6-c1a193cc9d10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:47.150874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50af850-f4ff-4d5f-b6e6-c1a193cc9d10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:47.152415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50af850-f4ff-4d5f-b6e6-c1a193cc9d10 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:47.175032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 336/685] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:20:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:47.175032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50af850-f4ff-4d5f-b6e6-c1a193cc9d10 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:47.198904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:47.253686 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:47.304468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 350/686] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:47.306339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813872cc-f5c1-4678-b236-95edf65eef4f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:47.318899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50af850-f4ff-4d5f-b6e6-c1a193cc9d10 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:47.357137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c8ed5-c82b-45af-981a-18a6b1dcad04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:47.357758 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c8ed5-c82b-45af-981a-18a6b1dcad04 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:47.358085 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c8ed5-c82b-45af-981a-18a6b1dcad04 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:47.358372 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c8ed5-c82b-45af-981a-18a6b1dcad04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:47.359163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c8ed5-c82b-45af-981a-18a6b1dcad04 None admin] RBAC: Adding query filter params (role.id=091d39715077464daa4cf890db55e6b9, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=f54b570990794801a5e45918845569b8) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:47.359473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c8ed5-c82b-45af-981a-18a6b1dcad04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:47.360941 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c8ed5-c82b-45af-981a-18a6b1dcad04 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:47.367307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 337/687] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:20:47 2022] GET /identity/v3/role_assignments?user.id=f54b570990794801a5e45918845569b8&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=091d39715077464daa4cf890db55e6b9 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:47.368144 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c8ed5-c82b-45af-981a-18a6b1dcad04 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:47.492488 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:47.500034 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813872cc-f5c1-4678-b236-95edf65eef4f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:47.528506 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c8ed5-c82b-45af-981a-18a6b1dcad04 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:47.554668 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64421bd-03b5-4f21-aa81-1e8859c7e15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:47.555596 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64421bd-03b5-4f21-aa81-1e8859c7e15f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:47.555905 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64421bd-03b5-4f21-aa81-1e8859c7e15f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:47.556181 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64421bd-03b5-4f21-aa81-1e8859c7e15f None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:47.556907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64421bd-03b5-4f21-aa81-1e8859c7e15f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:47.557232 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64421bd-03b5-4f21-aa81-1e8859c7e15f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:47.558689 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64421bd-03b5-4f21-aa81-1e8859c7e15f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:47.572991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 351/688] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:47.574411 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64421bd-03b5-4f21-aa81-1e8859c7e15f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:47.588948 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab64bf-67b7-4bcb-b997-5a71d93acac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:47.590109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab64bf-67b7-4bcb-b997-5a71d93acac3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:47.590584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab64bf-67b7-4bcb-b997-5a71d93acac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:47.591058 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab64bf-67b7-4bcb-b997-5a71d93acac3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:47.592003 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab64bf-67b7-4bcb-b997-5a71d93acac3 None admin] RBAC: Adding query filter params (role.id=091d39715077464daa4cf890db55e6b9, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=76311c7173b442c0af2c745e3e70a07d) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:47.592488 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab64bf-67b7-4bcb-b997-5a71d93acac3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:47.594081 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab64bf-67b7-4bcb-b997-5a71d93acac3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:47.601722 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 338/689] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:20:47 2022] GET /identity/v3/role_assignments?user.id=76311c7173b442c0af2c745e3e70a07d&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=091d39715077464daa4cf890db55e6b9 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:47.603444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9843543-8406-4a04-b015-f717a293c57f None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:47.603444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9843543-8406-4a04-b015-f717a293c57f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:47.603688 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9843543-8406-4a04-b015-f717a293c57f None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:47.605119 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 339/690] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:47.605813 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9843543-8406-4a04-b015-f717a293c57f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:47.633241 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:47.751309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64421bd-03b5-4f21-aa81-1e8859c7e15f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:47.799216 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b7782-2f9c-44f7-abd2-4c3d22ddff1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:47.800607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b7782-2f9c-44f7-abd2-4c3d22ddff1b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:47.801341 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b7782-2f9c-44f7-abd2-4c3d22ddff1b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:47.801750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b7782-2f9c-44f7-abd2-4c3d22ddff1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:47.807676 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9843543-8406-4a04-b015-f717a293c57f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:47.824146 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b7782-2f9c-44f7-abd2-4c3d22ddff1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:47.825809 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b7782-2f9c-44f7-abd2-4c3d22ddff1b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:47.844533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-b85b7782-2f9c-44f7-abd2-4c3d22ddff1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:47.844533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:47.844533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:47.844533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:47.844533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:47.844533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:47.844533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:47.844533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:47.844533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:47.844533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:47.844533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:47.844533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:47.844533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:47.844533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:47.844533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:47.845497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:47.845497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:47.845497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:47.845497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:47.845497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:47.845497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:47.845497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:47.845497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:47.845497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:47.845497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:47.845497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:47.845497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:47.845497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:47.845497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:47.845497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:47.845497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:47.845497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:47.846421 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:47.846421 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:47.846421 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:47.846421 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:47.846421 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 352/691] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:47.847167 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385cd2d-7776-4c21-b1fb-ebfb46eaa32b None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:47.847534 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385cd2d-7776-4c21-b1fb-ebfb46eaa32b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:47.847816 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385cd2d-7776-4c21-b1fb-ebfb46eaa32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:47.866006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f34444-94c0-4957-9979-2a95d8f36d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:47.867155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f34444-94c0-4957-9979-2a95d8f36d54 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:47.867636 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f34444-94c0-4957-9979-2a95d8f36d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:47.867741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c385cd2d-7776-4c21-b1fb-ebfb46eaa32b None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:47.868740 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f34444-94c0-4957-9979-2a95d8f36d54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:47.869463 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c385cd2d-7776-4c21-b1fb-ebfb46eaa32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:47.882365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:47.886815 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f34444-94c0-4957-9979-2a95d8f36d54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:47.890478 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f34444-94c0-4957-9979-2a95d8f36d54 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:47.914296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-20f34444-94c0-4957-9979-2a95d8f36d54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:47.914296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:47.914296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:47.914296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:47.914296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:47.914296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:47.914296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:47.914296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:47.914296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:47.914296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:47.914296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:47.914296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:47.914296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:47.914296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:47.914296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:47.915310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:47.915310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:47.915310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:47.915310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:47.915310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:47.915310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:47.915310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:47.915310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:47.915310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:47.915310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:47.915310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:47.915310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:47.915310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:47.915310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:47.915310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:47.915310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:47.915310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:47.916864 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:47.916864 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:47.916864 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:47.916864 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:47.916864 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 340/692] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:47.917209 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f34444-94c0-4957-9979-2a95d8f36d54 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:47.969021 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 353/693] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:47.970061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385cd2d-7776-4c21-b1fb-ebfb46eaa32b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:48.000114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:48.075428 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f34444-94c0-4957-9979-2a95d8f36d54 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:48.113487 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385cd2d-7776-4c21-b1fb-ebfb46eaa32b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:48.140692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9d0dc-1826-4258-9b9c-8fbeffbb5c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:48.141605 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9d0dc-1826-4258-9b9c-8fbeffbb5c3e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:48.142163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9d0dc-1826-4258-9b9c-8fbeffbb5c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:48.142584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9d0dc-1826-4258-9b9c-8fbeffbb5c3e None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:48.143863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9d0dc-1826-4258-9b9c-8fbeffbb5c3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:48.144302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9d0dc-1826-4258-9b9c-8fbeffbb5c3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:48.146993 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9d0dc-1826-4258-9b9c-8fbeffbb5c3e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:48.157445 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94eefe-f8b7-46f3-bca7-207210f0ac1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:48.158411 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94eefe-f8b7-46f3-bca7-207210f0ac1c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:48.159451 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94eefe-f8b7-46f3-bca7-207210f0ac1c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:48.159908 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94eefe-f8b7-46f3-bca7-207210f0ac1c None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:48.160764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94eefe-f8b7-46f3-bca7-207210f0ac1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:48.161223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94eefe-f8b7-46f3-bca7-207210f0ac1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:48.163331 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94eefe-f8b7-46f3-bca7-207210f0ac1c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:48.174310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 341/694] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:48.174310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9d0dc-1826-4258-9b9c-8fbeffbb5c3e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:48.188244 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 354/695] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:48.189687 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94eefe-f8b7-46f3-bca7-207210f0ac1c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:48.242554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:48.302828 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9d0dc-1826-4258-9b9c-8fbeffbb5c3e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:48.318874 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:48.346305 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd26222-1e1a-4b99-9164-31efd9342099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:48.347332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd26222-1e1a-4b99-9164-31efd9342099 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:48.347946 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd26222-1e1a-4b99-9164-31efd9342099 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:48.348525 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd26222-1e1a-4b99-9164-31efd9342099 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:48.348770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94eefe-f8b7-46f3-bca7-207210f0ac1c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:48.362124 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd26222-1e1a-4b99-9164-31efd9342099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:48.365426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd26222-1e1a-4b99-9164-31efd9342099 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:48.388937 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5db6dc-2298-45c6-90b6-8c4b999c9eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:48.389655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5db6dc-2298-45c6-90b6-8c4b999c9eea None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:48.389964 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5db6dc-2298-45c6-90b6-8c4b999c9eea None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:48.390268 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5db6dc-2298-45c6-90b6-8c4b999c9eea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:48.396805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-ddd26222-1e1a-4b99-9164-31efd9342099 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:48.396805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:48.396805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:48.396805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:48.396805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:48.396805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:48.396805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:48.396805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:48.396805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:48.396805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:48.396805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:48.396805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:48.396805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:48.396805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:48.396805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:48.398434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:48.398434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:48.398434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:48.398434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:48.398434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:48.398434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:48.398434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:48.398434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:48.398434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:48.398434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:48.398434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:48.398434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:48.398434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:48.398434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:48.398434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:48.398434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:48.398434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:48.400196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:48.400196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:48.400196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:48.400196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:48.400612 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5db6dc-2298-45c6-90b6-8c4b999c9eea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:48.400612 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5db6dc-2298-45c6-90b6-8c4b999c9eea None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:48.404674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 342/696] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:48.406574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd26222-1e1a-4b99-9164-31efd9342099 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:48.418415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-9b5db6dc-2298-45c6-90b6-8c4b999c9eea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:48.418415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:48.418415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:48.418415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:48.418415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:48.418415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:48.418415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:48.418415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:48.418415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:48.418415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:48.418415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:48.418415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:48.418415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:20:48.418415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:20:48.418415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:20:48.419397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:20:48.419397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:48.419397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:48.419397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:48.419397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:48.419397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:48.419397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:48.419397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:48.419397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:48.419397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:48.419397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:48.419397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:48.419397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:48.419397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:48.419397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:48.419397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:48.419397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:48.420330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:20:48.420330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:20:48.420330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:20:48.420330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:20:48.420330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:20:48.420330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:20:48.420330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:48.420330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:48.424562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 355/697] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:48.425440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5db6dc-2298-45c6-90b6-8c4b999c9eea None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:48.542359 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5db6dc-2298-45c6-90b6-8c4b999c9eea None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:48.554506 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd26222-1e1a-4b99-9164-31efd9342099 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:48.557878 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:48.582615 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd88c4-a6d1-4621-9fa6-9e7d7c3a73d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:48.583347 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd88c4-a6d1-4621-9fa6-9e7d7c3a73d6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:48.583636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd88c4-a6d1-4621-9fa6-9e7d7c3a73d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:48.583909 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd88c4-a6d1-4621-9fa6-9e7d7c3a73d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:48.584618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd88c4-a6d1-4621-9fa6-9e7d7c3a73d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:48.584991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd88c4-a6d1-4621-9fa6-9e7d7c3a73d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:48.586514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd88c4-a6d1-4621-9fa6-9e7d7c3a73d6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:48.588768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:48.598673 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1243f-a202-42ac-840e-1476da45e359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:48.599687 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1243f-a202-42ac-840e-1476da45e359 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:48.600180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1243f-a202-42ac-840e-1476da45e359 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:48.600681 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1243f-a202-42ac-840e-1476da45e359 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:48.606350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 356/698] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:48.607286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd88c4-a6d1-4621-9fa6-9e7d7c3a73d6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:48.612328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1243f-a202-42ac-840e-1476da45e359 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:48.614258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1243f-a202-42ac-840e-1476da45e359 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:48.627412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-d2f1243f-a202-42ac-840e-1476da45e359 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:20:48.627412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:48.627412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:48.627412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:48.627412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:48.627412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:48.627412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:48.627412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:48.627412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:48.627412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:48.627412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:48.627412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:48.627412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:20:48.627412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:20:48.627412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:20:48.627412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:20:48.628423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:48.628423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:48.628423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:20:48.628423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:48.628423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:20:48.628423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:48.628423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:48.628423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:48.628423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:48.628423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:48.628423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:48.628423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:48.628423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:48.628423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:48.628423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:48.628423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:48.628423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:48.628423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:48.629379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:48.629379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:48.629379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:20:48.629379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:20:48.629379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:20:48.629379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:20:48.629379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:20:48.629379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:20:48.629379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:20:48.629379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:48.632197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 343/699] 10.222.0.83 () {56 vars in 1093 bytes} [Mon Aug 1 15:20:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:48.633608 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1243f-a202-42ac-840e-1476da45e359 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:48.730954 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd88c4-a6d1-4621-9fa6-9e7d7c3a73d6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:48.761329 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1243f-a202-42ac-840e-1476da45e359 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:48.767732 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbeaa2-9e78-4f4c-af79-792b06d3913b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:48.768366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbeaa2-9e78-4f4c-af79-792b06d3913b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:48.768749 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbeaa2-9e78-4f4c-af79-792b06d3913b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:48.769032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbeaa2-9e78-4f4c-af79-792b06d3913b None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:48.769784 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dbeaa2-9e78-4f4c-af79-792b06d3913b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:48.770138 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dbeaa2-9e78-4f4c-af79-792b06d3913b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:48.771673 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dbeaa2-9e78-4f4c-af79-792b06d3913b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:48.789877 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 357/700] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:20:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:48.791425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbeaa2-9e78-4f4c-af79-792b06d3913b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:48.799137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4b91b-0048-42aa-953c-2bdcfe0e479c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:48.799863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4b91b-0048-42aa-953c-2bdcfe0e479c None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:48.800152 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4b91b-0048-42aa-953c-2bdcfe0e479c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:48.800425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4b91b-0048-42aa-953c-2bdcfe0e479c None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:48.801199 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4b91b-0048-42aa-953c-2bdcfe0e479c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:48.802792 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4b91b-0048-42aa-953c-2bdcfe0e479c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:48.830336 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 344/701] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Aug 1 15:20:48 2022] POST /identity/v3/users => generated 254 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:20:48.831530 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4b91b-0048-42aa-953c-2bdcfe0e479c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:48.897149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:48.898318 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:49.014178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbeaa2-9e78-4f4c-af79-792b06d3913b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:49.068064 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4b91b-0048-42aa-953c-2bdcfe0e479c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:49.097318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7821796-7f97-4a01-8c5d-b3c712f287d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:49.097990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7821796-7f97-4a01-8c5d-b3c712f287d7 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:49.098321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7821796-7f97-4a01-8c5d-b3c712f287d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:49.098597 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7821796-7f97-4a01-8c5d-b3c712f287d7 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:49.099403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7821796-7f97-4a01-8c5d-b3c712f287d7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:49.099706 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7821796-7f97-4a01-8c5d-b3c712f287d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:49.101147 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7821796-7f97-4a01-8c5d-b3c712f287d7 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:49.125102 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 358/702] 10.222.0.83 () {56 vars in 1133 bytes} [Mon Aug 1 15:20:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:49.126217 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7212c-0f49-42a9-96e4-2193bb1407fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:49.126853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7212c-0f49-42a9-96e4-2193bb1407fb None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:49.127134 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7212c-0f49-42a9-96e4-2193bb1407fb None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:49.127406 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7212c-0f49-42a9-96e4-2193bb1407fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:49.128202 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7212c-0f49-42a9-96e4-2193bb1407fb None admin] RBAC: Adding query filter params (role.id=091d39715077464daa4cf890db55e6b9, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=6873a3e440bd4b63b0082fc85beef49c) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:49.128507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7212c-0f49-42a9-96e4-2193bb1407fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:49.130062 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7212c-0f49-42a9-96e4-2193bb1407fb None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:49.134260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7821796-7f97-4a01-8c5d-b3c712f287d7 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:49.143917 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 345/703] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:20:48 2022] GET /identity/v3/role_assignments?user.id=6873a3e440bd4b63b0082fc85beef49c&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=091d39715077464daa4cf890db55e6b9 => generated 608 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:49.305382 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7821796-7f97-4a01-8c5d-b3c712f287d7 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:49.331588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:49.370831 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadd787-75e3-4d0e-b6d9-f6dd3f08e163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:49.372273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadd787-75e3-4d0e-b6d9-f6dd3f08e163 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:49.372940 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadd787-75e3-4d0e-b6d9-f6dd3f08e163 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:49.373596 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadd787-75e3-4d0e-b6d9-f6dd3f08e163 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:49.502303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadd787-75e3-4d0e-b6d9-f6dd3f08e163 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:49.505459 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadd787-75e3-4d0e-b6d9-f6dd3f08e163 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:49.694033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 359/704] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:49.703344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7212c-0f49-42a9-96e4-2193bb1407fb None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:49.835362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:49.854746 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7212c-0f49-42a9-96e4-2193bb1407fb None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:49.901135 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aff854-8b02-4d01-994b-46da6dc27a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:49.901797 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aff854-8b02-4d01-994b-46da6dc27a15 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:49.902115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aff854-8b02-4d01-994b-46da6dc27a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:49.902407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aff854-8b02-4d01-994b-46da6dc27a15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:49.916142 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aff854-8b02-4d01-994b-46da6dc27a15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:49.917644 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aff854-8b02-4d01-994b-46da6dc27a15 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:49.934933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-78aff854-8b02-4d01-994b-46da6dc27a15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:49.934933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:49.934933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:49.934933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:49.934933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:49.934933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:49.934933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:49.934933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:49.934933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:49.934933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:49.934933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:49.934933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:49.934933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:49.934933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:49.934933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:49.936680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:49.936680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:49.936680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:49.936680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:49.936680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:49.936680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:49.936680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:49.936680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:49.936680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:49.936680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:49.936680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:49.936680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:49.936680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:49.936680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:49.936680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:49.936680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:49.936680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:49.938460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:49.938460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:49.938460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:49.938460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:49.938460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 346/705] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:49.948656 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadd787-75e3-4d0e-b6d9-f6dd3f08e163 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:49.964685 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:50.079237 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e9cc3-8d19-422a-a34e-7050bf1e2db1 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:50.079527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e9cc3-8d19-422a-a34e-7050bf1e2db1 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:50.079826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e9cc3-8d19-422a-a34e-7050bf1e2db1 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:50.081114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 347/706] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:50.087517 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadd787-75e3-4d0e-b6d9-f6dd3f08e163 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:50.088194 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ea70b-39ef-42ac-95d4-a4e12d682dcd None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:50.088489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ea70b-39ef-42ac-95d4-a4e12d682dcd None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:50.088770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ea70b-39ef-42ac-95d4-a4e12d682dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:50.123721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-697ea70b-39ef-42ac-95d4-a4e12d682dcd None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:50.124976 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-697ea70b-39ef-42ac-95d4-a4e12d682dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:50.158675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d6e52-bae0-454b-8f6b-721689232dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:50.159378 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d6e52-bae0-454b-8f6b-721689232dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:50.159665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d6e52-bae0-454b-8f6b-721689232dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:50.159942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d6e52-bae0-454b-8f6b-721689232dc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:50.160661 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d6e52-bae0-454b-8f6b-721689232dc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:50.160969 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d6e52-bae0-454b-8f6b-721689232dc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:50.162502 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d6e52-bae0-454b-8f6b-721689232dc9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:50.176766 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 360/707] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:50.182015 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618916e-cd4d-4c29-9050-7f9c300a8047 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:50.182015 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618916e-cd4d-4c29-9050-7f9c300a8047 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:50.182015 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618916e-cd4d-4c29-9050-7f9c300a8047 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:50.182015 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 361/708] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:50.186719 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618916e-cd4d-4c29-9050-7f9c300a8047 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:50.266613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:50.268672 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 348/709] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:50.270433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8996e4-705c-4ac8-913e-029fefca7bc7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:50.270946 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8996e4-705c-4ac8-913e-029fefca7bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:50.271422 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8996e4-705c-4ac8-913e-029fefca7bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:50.292508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4a8996e4-705c-4ac8-913e-029fefca7bc7 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:50.294222 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4a8996e4-705c-4ac8-913e-029fefca7bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:50.304588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618916e-cd4d-4c29-9050-7f9c300a8047 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:50.347983 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283cd2c-3f2b-415d-a0b3-1bf0534a0205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:50.348677 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283cd2c-3f2b-415d-a0b3-1bf0534a0205 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:50.348936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283cd2c-3f2b-415d-a0b3-1bf0534a0205 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:50.349208 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283cd2c-3f2b-415d-a0b3-1bf0534a0205 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:50.357649 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283cd2c-3f2b-415d-a0b3-1bf0534a0205 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:50.359188 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283cd2c-3f2b-415d-a0b3-1bf0534a0205 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:50.371588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-b283cd2c-3f2b-415d-a0b3-1bf0534a0205 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:50.371588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:50.371588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:50.371588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:50.371588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:50.371588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:50.371588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:50.371588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:50.371588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:50.371588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:50.371588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:50.371588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:50.371588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:20:50.371588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:20:50.371588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:20:50.373303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:20:50.373303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:50.373303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:50.373303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:50.373303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:50.373303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:50.373303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:50.373303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:50.373303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:50.373303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:50.373303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:50.373303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:50.373303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:50.373303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:50.373303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:50.373303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:50.373303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:50.375101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:20:50.375101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:20:50.375101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:20:50.375101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:20:50.375101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:20:50.375101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:20:50.375101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:50.375101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:50.375101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 362/710] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:50.375101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283cd2c-3f2b-415d-a0b3-1bf0534a0205 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:50.401932 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:50.402515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 349/711] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:50.404257 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8996e4-705c-4ac8-913e-029fefca7bc7 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:50.494651 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283cd2c-3f2b-415d-a0b3-1bf0534a0205 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:50.527112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:50.533235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52955d6e-2321-4b5c-b3b8-a3feaaa0c4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:50.534102 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52955d6e-2321-4b5c-b3b8-a3feaaa0c4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:50.534394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52955d6e-2321-4b5c-b3b8-a3feaaa0c4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:50.534694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52955d6e-2321-4b5c-b3b8-a3feaaa0c4f9 None admin] PATH_INFO: `/v3/services/network` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:50.535306 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52955d6e-2321-4b5c-b3b8-a3feaaa0c4f9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:50.536847 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52955d6e-2321-4b5c-b3b8-a3feaaa0c4f9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:50.549436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-52955d6e-2321-4b5c-b3b8-a3feaaa0c4f9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:20:50.549436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:50.549436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:20:50.549436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:20:50.549436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:20:50.549436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:20:50.549436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:20:50.549436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:20:50.549436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:20:50.549436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:50.549436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:20:50.549436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:50.549436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:50.549436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:50.549436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:50.549436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:50.549436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:50.551469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:50.551469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:50.551469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:50.551469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:50.551469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:50.551469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:50.551469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:20:50.551469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:20:50.551469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:20:50.551469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:20:50.551469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:50.551469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:50.551469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:50.551469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:50.551469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:50.551469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:50.553141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:50.553141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:50.553141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:50.553141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:50.553141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:50.553141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:50.553141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:50.553141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:50.553141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:50.553141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:50.553141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:20:50.553141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:20:50.553141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:20:50.553141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:50.553141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 363/712] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:50 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:50.557517 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52955d6e-2321-4b5c-b3b8-a3feaaa0c4f9 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:50.630540 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8996e4-705c-4ac8-913e-029fefca7bc7 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:50.676275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201632db-8a5e-48eb-ad8c-8559f68a179a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:50.677009 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201632db-8a5e-48eb-ad8c-8559f68a179a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:50.677298 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201632db-8a5e-48eb-ad8c-8559f68a179a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:50.677577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201632db-8a5e-48eb-ad8c-8559f68a179a None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:50.678476 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201632db-8a5e-48eb-ad8c-8559f68a179a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:50.678790 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201632db-8a5e-48eb-ad8c-8559f68a179a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:50.680331 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201632db-8a5e-48eb-ad8c-8559f68a179a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:50.700151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 350/713] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:20:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:50.701092 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201632db-8a5e-48eb-ad8c-8559f68a179a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:50.725410 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52955d6e-2321-4b5c-b3b8-a3feaaa0c4f9 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:50.778097 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48294218-8b3c-446b-bb9b-d00bb370a402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:50.779482 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48294218-8b3c-446b-bb9b-d00bb370a402 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:50.780175 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48294218-8b3c-446b-bb9b-d00bb370a402 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:50.780847 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48294218-8b3c-446b-bb9b-d00bb370a402 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:50.782296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48294218-8b3c-446b-bb9b-d00bb370a402 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:50.785075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48294218-8b3c-446b-bb9b-d00bb370a402 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:50.801286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-48294218-8b3c-446b-bb9b-d00bb370a402 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:20:50.801286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:50.801286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:20:50.801286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:20:50.801286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:20:50.801286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:20:50.801286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:20:50.801286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:20:50.801286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:20:50.801286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:50.801286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:20:50.801286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:50.801286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:50.801286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:50.801286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:50.801286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:50.802872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:50.802872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:50.802872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:50.802872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:50.802872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:50.802872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:50.802872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:50.802872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:20:50.802872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:20:50.802872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:20:50.802872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:20:50.802872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:50.802872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:50.802872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:50.802872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:50.802872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:50.804309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:50.804309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:50.804309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:50.804309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:50.804309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:50.804309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:50.804309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:50.804309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:50.804309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:50.804309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:50.804309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:50.804309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:20:50.804309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:20:50.804309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:20:50.804309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:50.805693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:50.816488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 364/714] 10.222.0.83 () {56 vars in 1129 bytes} [Mon Aug 1 15:20:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:50.818401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48294218-8b3c-446b-bb9b-d00bb370a402 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:50.847914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201632db-8a5e-48eb-ad8c-8559f68a179a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:50.885451 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211cc27-c5d6-4866-9f59-9445e1931970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:50.886125 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211cc27-c5d6-4866-9f59-9445e1931970 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:50.886413 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211cc27-c5d6-4866-9f59-9445e1931970 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:50.886796 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211cc27-c5d6-4866-9f59-9445e1931970 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:50.887872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211cc27-c5d6-4866-9f59-9445e1931970 None admin] RBAC: Adding query filter params (name=network) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:50.888003 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211cc27-c5d6-4866-9f59-9445e1931970 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:50.890214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211cc27-c5d6-4866-9f59-9445e1931970 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:50.898056 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 351/715] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Aug 1 15:20:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:50.899112 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211cc27-c5d6-4866-9f59-9445e1931970 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:50.905452 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:50.971433 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48294218-8b3c-446b-bb9b-d00bb370a402 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:51.033648 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc799729-01b0-4a70-9b66-97817a36bb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:51.035592 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc799729-01b0-4a70-9b66-97817a36bb94 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:51.036824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc799729-01b0-4a70-9b66-97817a36bb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:51.037958 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc799729-01b0-4a70-9b66-97817a36bb94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:51.039603 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc799729-01b0-4a70-9b66-97817a36bb94 None admin] RBAC: Adding query filter params (role.id=7bdfa43dc1d046f7965ffe52b74bc58f, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=25dc41ffc41c47fdbeec06110d04610c) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:51.040603 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc799729-01b0-4a70-9b66-97817a36bb94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:51.044379 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc799729-01b0-4a70-9b66-97817a36bb94 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:51.056121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 365/716] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:20:50 2022] GET /identity/v3/role_assignments?user.id=25dc41ffc41c47fdbeec06110d04610c&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=7bdfa43dc1d046f7965ffe52b74bc58f => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:51.058509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc799729-01b0-4a70-9b66-97817a36bb94 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:51.124118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211cc27-c5d6-4866-9f59-9445e1931970 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:51.170905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:51.203121 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd17df9-a193-4b12-b310-629412fad659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:51.204531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd17df9-a193-4b12-b310-629412fad659 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:51.205234 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd17df9-a193-4b12-b310-629412fad659 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:51.205968 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd17df9-a193-4b12-b310-629412fad659 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:51.207843 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd17df9-a193-4b12-b310-629412fad659 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:51.208670 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd17df9-a193-4b12-b310-629412fad659 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:51.211905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd17df9-a193-4b12-b310-629412fad659 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:51.222206 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 352/717] 10.222.0.83 () {56 vars in 1124 bytes} [Mon Aug 1 15:20:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:51.230668 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd17df9-a193-4b12-b310-629412fad659 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:51.339561 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc799729-01b0-4a70-9b66-97817a36bb94 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:51.395737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:51.417325 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c59280-bef3-4bc1-a87f-e37897c5542c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:51.418183 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c59280-bef3-4bc1-a87f-e37897c5542c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:51.418524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c59280-bef3-4bc1-a87f-e37897c5542c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:51.418802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c59280-bef3-4bc1-a87f-e37897c5542c None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:51.419501 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c59280-bef3-4bc1-a87f-e37897c5542c None admin] RBAC: Adding query filter params (type=network) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:51.420078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c59280-bef3-4bc1-a87f-e37897c5542c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:51.422337 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd17df9-a193-4b12-b310-629412fad659 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:51.425488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c59280-bef3-4bc1-a87f-e37897c5542c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:51.440528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 366/718] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Aug 1 15:20:51 2022] GET /identity/v3/services?type=network => generated 124 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:51.473107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9046a9-20b9-4cba-90f3-6d763e3d62e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:51.473985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9046a9-20b9-4cba-90f3-6d763e3d62e6 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:51.474376 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9046a9-20b9-4cba-90f3-6d763e3d62e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:51.474754 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9046a9-20b9-4cba-90f3-6d763e3d62e6 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:51.475574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9046a9-20b9-4cba-90f3-6d763e3d62e6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:51.476011 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9046a9-20b9-4cba-90f3-6d763e3d62e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:51.478198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9046a9-20b9-4cba-90f3-6d763e3d62e6 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:51.485590 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 353/719] 10.222.0.83 () {56 vars in 1124 bytes} [Mon Aug 1 15:20:51 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:51.604659 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa73e7-c9f0-4056-9524-fa446624ed30 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:51.604873 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa73e7-c9f0-4056-9524-fa446624ed30 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:51.605218 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa73e7-c9f0-4056-9524-fa446624ed30 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:51.606690 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 367/720] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:20:51.614270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bade23-6aba-4f3b-ab1a-882a5bf43149 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:51.614637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bade23-6aba-4f3b-ab1a-882a5bf43149 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:51.614862 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bade23-6aba-4f3b-ab1a-882a5bf43149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:51.643204 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-61bade23-6aba-4f3b-ab1a-882a5bf43149 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:51.644438 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-61bade23-6aba-4f3b-ab1a-882a5bf43149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:51.711499 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:51.756572 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 354/721] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:51.786714 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa73e7-c9f0-4056-9524-fa446624ed30 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:51.920260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa73e7-c9f0-4056-9524-fa446624ed30 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:51.963260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc59594-a407-4e1c-a730-d25a88266afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:51.963983 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc59594-a407-4e1c-a730-d25a88266afa None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:51.964363 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc59594-a407-4e1c-a730-d25a88266afa None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:51.964826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc59594-a407-4e1c-a730-d25a88266afa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:51.978208 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc59594-a407-4e1c-a730-d25a88266afa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:51.984139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-5dc59594-a407-4e1c-a730-d25a88266afa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:51.984139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:51.984139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:51.984139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:51.984139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:51.984139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:51.984139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:51.984139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:51.984139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:51.984139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:51.984139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:51.984139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:51.984139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:51.984139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:51.984139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:51.984139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:51.985559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:51.985559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:51.985559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:51.985559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:51.985559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:51.985559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:51.985559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:51.985559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:51.985559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:51.985559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:51.985559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:51.985559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:51.985559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:51.985559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:51.985559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:51.985559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:51.986942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:51.986942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:51.986942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:51.986942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:51.986942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:51.986942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:51.986942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:51.986942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:51.986942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:51.986942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:51.986942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:51.986942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:51.986942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 368/722] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:20:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:51.992262 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bade23-6aba-4f3b-ab1a-882a5bf43149 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:52.208581 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bade23-6aba-4f3b-ab1a-882a5bf43149 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:52.270086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c697699-dd10-44fe-a6de-d03b01144fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:52.270725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c697699-dd10-44fe-a6de-d03b01144fcd None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:52.271009 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c697699-dd10-44fe-a6de-d03b01144fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:52.271393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c697699-dd10-44fe-a6de-d03b01144fcd None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:52.272494 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c697699-dd10-44fe-a6de-d03b01144fcd None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:52.272802 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c697699-dd10-44fe-a6de-d03b01144fcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:52.274283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c697699-dd10-44fe-a6de-d03b01144fcd None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:52.285559 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 355/723] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:20:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:52.291543 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc59594-a407-4e1c-a730-d25a88266afa None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:52.362398 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:52.413613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc59594-a407-4e1c-a730-d25a88266afa None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:52.452483 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb607dd-dd5a-492f-adcc-512f92c9e3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:52.453214 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb607dd-dd5a-492f-adcc-512f92c9e3eb None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:52.453837 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb607dd-dd5a-492f-adcc-512f92c9e3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:52.454145 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb607dd-dd5a-492f-adcc-512f92c9e3eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:52.466552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb607dd-dd5a-492f-adcc-512f92c9e3eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:52.467934 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb607dd-dd5a-492f-adcc-512f92c9e3eb None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:52.484075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-bbb607dd-dd5a-492f-adcc-512f92c9e3eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:52.484075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:52.484075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:52.484075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:52.484075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:52.484075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:52.484075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:52.484075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:52.484075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:52.484075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:52.484075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:52.484075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:52.484075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:52.484075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:52.484075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:52.485528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:52.485528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:52.485528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:52.485528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:52.485528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:52.485528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:52.485528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:52.485528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:52.485528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:52.485528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:52.485528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:52.485528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:52.485528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:52.485528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:52.485528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:52.485528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:52.485528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:52.487087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:52.487087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:52.487087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:52.487087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:52.487087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 369/724] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:52.493394 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c697699-dd10-44fe-a6de-d03b01144fcd None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:52.554400 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:52.583485 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45bc50b-9238-40e7-9167-3a83b2444686 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:52.583984 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45bc50b-9238-40e7-9167-3a83b2444686 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:52.584470 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45bc50b-9238-40e7-9167-3a83b2444686 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:52.587028 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 370/725] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:20:52.593938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5cdb2-aacb-4c31-b953-91d9e0976e61 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:52.594261 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5cdb2-aacb-4c31-b953-91d9e0976e61 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:52.594488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5cdb2-aacb-4c31-b953-91d9e0976e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:52.628692 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-46a5cdb2-aacb-4c31-b953-91d9e0976e61 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:52.630399 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-46a5cdb2-aacb-4c31-b953-91d9e0976e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:52.677938 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c697699-dd10-44fe-a6de-d03b01144fcd None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:52.722816 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fee80-36fb-4f65-9352-93c3e36d436e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:52.723372 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fee80-36fb-4f65-9352-93c3e36d436e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:52.723657 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fee80-36fb-4f65-9352-93c3e36d436e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:52.723934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fee80-36fb-4f65-9352-93c3e36d436e None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:52.724657 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fee80-36fb-4f65-9352-93c3e36d436e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:52.724967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fee80-36fb-4f65-9352-93c3e36d436e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:52.726439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fee80-36fb-4f65-9352-93c3e36d436e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:52.759830 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 356/726] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:52.766500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fee80-36fb-4f65-9352-93c3e36d436e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:52.806515 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 371/727] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:52.852779 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5cdb2-aacb-4c31-b953-91d9e0976e61 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:52.925630 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:52.946726 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:52.951336 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fee80-36fb-4f65-9352-93c3e36d436e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:53.001290 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce35aa-3323-48c0-a9d3-5b93b253e55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:53.002548 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce35aa-3323-48c0-a9d3-5b93b253e55e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:53.004751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce35aa-3323-48c0-a9d3-5b93b253e55e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:53.005189 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce35aa-3323-48c0-a9d3-5b93b253e55e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:53.029133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce35aa-3323-48c0-a9d3-5b93b253e55e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:53.031516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce35aa-3323-48c0-a9d3-5b93b253e55e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:53.075643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-13ce35aa-3323-48c0-a9d3-5b93b253e55e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:20:53.075643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:53.075643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:53.075643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:53.075643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:53.075643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:53.075643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:53.075643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:53.075643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:53.075643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:53.075643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:53.075643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:53.075643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:20:53.075643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:20:53.075643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:20:53.075643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:20:53.077165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:53.077165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:53.077165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:20:53.077165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:53.077165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:20:53.077165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:53.077165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:53.077165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:53.077165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:53.077165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:53.077165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:53.077165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:53.077165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:53.077165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:53.077165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:53.077165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:53.077165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:53.077165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:53.078697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:53.078697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:53.078697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:20:53.078697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:20:53.078697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:20:53.078697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:20:53.078697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:20:53.078697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:20:53.078697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:20:53.078697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:53.078697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 357/728] 10.222.0.83 () {56 vars in 1117 bytes} [Mon Aug 1 15:20:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:53.087718 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce35aa-3323-48c0-a9d3-5b93b253e55e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:53.110381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5cdb2-aacb-4c31-b953-91d9e0976e61 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:53.174939 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1a42b-b705-472b-94d1-fa42f99c6b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:53.175924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1a42b-b705-472b-94d1-fa42f99c6b1f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:53.176365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1a42b-b705-472b-94d1-fa42f99c6b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:53.176777 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1a42b-b705-472b-94d1-fa42f99c6b1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:53.219150 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1a42b-b705-472b-94d1-fa42f99c6b1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:53.235635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-4dc1a42b-b705-472b-94d1-fa42f99c6b1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:20:53.235635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:53.235635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:53.235635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:53.235635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:53.235635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:53.235635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:53.235635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:53.235635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:53.235635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:53.235635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:53.235635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:53.235635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:53.235635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:53.235635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:53.235635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:53.237116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:53.237116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:53.237116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:53.237116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:53.237116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:53.237116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:53.237116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:53.237116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:53.237116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:53.237116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:53.237116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:53.237116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:53.237116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:53.237116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:53.237116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:53.237116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:53.238447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:53.238447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:53.238447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:53.238447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:53.238447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:53.238447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:53.238447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:53.238447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:53.238447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:53.238447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:53.238447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:20:53.238447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:53.238447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 372/729] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:20:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:20:53.241924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce35aa-3323-48c0-a9d3-5b93b253e55e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:53.249279 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1a42b-b705-472b-94d1-fa42f99c6b1f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:53.285260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafd8da-c284-437a-b8d3-54fdf62b542c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:53.285968 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafd8da-c284-437a-b8d3-54fdf62b542c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:53.286256 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafd8da-c284-437a-b8d3-54fdf62b542c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:53.286532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafd8da-c284-437a-b8d3-54fdf62b542c None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:53.287343 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccafd8da-c284-437a-b8d3-54fdf62b542c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:53.287649 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccafd8da-c284-437a-b8d3-54fdf62b542c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:53.289091 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccafd8da-c284-437a-b8d3-54fdf62b542c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:53.326392 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 358/730] 10.222.0.83 () {56 vars in 1149 bytes} [Mon Aug 1 15:20:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:53.334981 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafd8da-c284-437a-b8d3-54fdf62b542c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:53.384576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:53.424680 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:53.441510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1a42b-b705-472b-94d1-fa42f99c6b1f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:53.492997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09ed12-f94c-4b2d-ab30-e5ade0407de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:53.494370 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09ed12-f94c-4b2d-ab30-e5ade0407de4 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:53.494883 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09ed12-f94c-4b2d-ab30-e5ade0407de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:53.495242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09ed12-f94c-4b2d-ab30-e5ade0407de4 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:53.497121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b09ed12-f94c-4b2d-ab30-e5ade0407de4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:53.497451 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b09ed12-f94c-4b2d-ab30-e5ade0407de4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:53.499015 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b09ed12-f94c-4b2d-ab30-e5ade0407de4 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:53.521412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 373/731] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:20:53.528617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09ed12-f94c-4b2d-ab30-e5ade0407de4 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:53.563617 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafd8da-c284-437a-b8d3-54fdf62b542c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:53.645444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d713c-b155-4893-897b-8f5315de4485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:53.646197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d713c-b155-4893-897b-8f5315de4485 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:53.646485 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d713c-b155-4893-897b-8f5315de4485 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:53.646772 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d713c-b155-4893-897b-8f5315de4485 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:53.734956 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09ed12-f94c-4b2d-ab30-e5ade0407de4 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:53.804673 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e601f-ed9a-4904-aa52-deb968fe3969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:53.805942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e601f-ed9a-4904-aa52-deb968fe3969 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:53.806447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e601f-ed9a-4904-aa52-deb968fe3969 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:53.806945 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e601f-ed9a-4904-aa52-deb968fe3969 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:53.820467 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d713c-b155-4893-897b-8f5315de4485 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:53.822123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d713c-b155-4893-897b-8f5315de4485 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:53.829391 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e601f-ed9a-4904-aa52-deb968fe3969 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:53.833202 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e601f-ed9a-4904-aa52-deb968fe3969 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:53.866360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-427e601f-ed9a-4904-aa52-deb968fe3969 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:53.866360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:53.866360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:53.866360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:53.866360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:53.866360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:53.866360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:53.866360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:53.866360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:53.866360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:53.866360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:53.866360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:53.866360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:53.866360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:53.866360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:53.867865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:53.867865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:53.867865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:53.867865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:53.867865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:53.867865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:53.867865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:53.867865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:53.867865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:53.867865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:53.867865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:53.867865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:53.867865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:53.867865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:53.867865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:53.867865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:53.867865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:53.869258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:53.869258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:53.869258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:53.869258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:53.869607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 374/732] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:53.871781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:53.879659 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e601f-ed9a-4904-aa52-deb968fe3969 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:53.886388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:54.021534 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 359/733] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:54.028160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d713c-b155-4893-897b-8f5315de4485 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:54.160703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e601f-ed9a-4904-aa52-deb968fe3969 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:54.181242 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d713c-b155-4893-897b-8f5315de4485 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:54.207792 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:54.222703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b629b41-3aa1-41cc-9706-48ceab55dfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:54.225598 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b629b41-3aa1-41cc-9706-48ceab55dfa5 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:54.226231 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b629b41-3aa1-41cc-9706-48ceab55dfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:54.226927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b629b41-3aa1-41cc-9706-48ceab55dfa5 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:54.228635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b629b41-3aa1-41cc-9706-48ceab55dfa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:54.229151 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b629b41-3aa1-41cc-9706-48ceab55dfa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:54.231786 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b629b41-3aa1-41cc-9706-48ceab55dfa5 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:54.238416 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a20481-46a1-4eee-806b-af4481424e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:54.239729 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a20481-46a1-4eee-806b-af4481424e93 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:54.240414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a20481-46a1-4eee-806b-af4481424e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:54.241109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a20481-46a1-4eee-806b-af4481424e93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:54.253695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 375/734] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:54.255985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefab0cd-dab0-463e-a1b7-6b08653be984 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:54.256424 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefab0cd-dab0-463e-a1b7-6b08653be984 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:54.256851 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefab0cd-dab0-463e-a1b7-6b08653be984 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:54.258892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a20481-46a1-4eee-806b-af4481424e93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:54.264691 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 376/735] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:54.264691 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefab0cd-dab0-463e-a1b7-6b08653be984 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:54.267129 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a20481-46a1-4eee-806b-af4481424e93 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:54.291049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-75a20481-46a1-4eee-806b-af4481424e93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:54.291049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:54.291049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:54.291049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:54.291049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:54.291049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:54.291049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:54.291049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:54.291049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:54.291049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:54.291049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:54.291049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:54.291049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:54.291049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:54.291049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:54.292170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:54.292170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:54.292170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:54.292170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:54.292170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:54.292170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:54.292170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:54.292170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:54.292170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:54.292170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:54.292170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:54.292170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:54.292170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:54.292170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:54.292170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:54.292170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:54.292170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:54.293170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:54.293170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:54.293170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:54.293170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:54.295701 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 360/736] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:54.297436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea2dec-e224-4a16-891d-4acaba20bfc0 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:54.297986 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea2dec-e224-4a16-891d-4acaba20bfc0 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:54.298563 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea2dec-e224-4a16-891d-4acaba20bfc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:54.319613 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d9ea2dec-e224-4a16-891d-4acaba20bfc0 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:54.321837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d9ea2dec-e224-4a16-891d-4acaba20bfc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:54.376781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:54.408053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 361/737] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:54.409465 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea2dec-e224-4a16-891d-4acaba20bfc0 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:54.451502 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefab0cd-dab0-463e-a1b7-6b08653be984 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:54.517499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde88508-2488-4aac-9114-9accf6318c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:54.518511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde88508-2488-4aac-9114-9accf6318c0b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:54.518957 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde88508-2488-4aac-9114-9accf6318c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:54.519377 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde88508-2488-4aac-9114-9accf6318c0b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:54.535632 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde88508-2488-4aac-9114-9accf6318c0b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:54.538218 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde88508-2488-4aac-9114-9accf6318c0b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:54.568283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-cde88508-2488-4aac-9114-9accf6318c0b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:20:54.568283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:54.568283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:54.568283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:54.568283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:54.568283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:54.568283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:54.568283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:54.568283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:54.568283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:54.568283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:54.568283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:54.568283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:20:54.568283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:20:54.568283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:20:54.568283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:20:54.570064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:54.570064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:54.570064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:20:54.570064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:54.570064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:20:54.570064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:54.570064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:54.570064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:54.570064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:54.570064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:54.570064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:54.570064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:54.570064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:54.570064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:54.570064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:54.570064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:54.570064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:54.570064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:54.572601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:54.572601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:54.572601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:20:54.572601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:20:54.572601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:20:54.572601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:20:54.572601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:20:54.572601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:20:54.572601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:20:54.572601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:54.576933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 377/738] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:20:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:54.578703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde88508-2488-4aac-9114-9accf6318c0b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:54.587520 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:54.597669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea2dec-e224-4a16-891d-4acaba20bfc0 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:54.678412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfff78-4167-4baa-8fc3-66d7a52860ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:54.679495 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfff78-4167-4baa-8fc3-66d7a52860ab None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:54.680002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfff78-4167-4baa-8fc3-66d7a52860ab None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:54.680571 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfff78-4167-4baa-8fc3-66d7a52860ab None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:54.684584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfff78-4167-4baa-8fc3-66d7a52860ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:54.684988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfff78-4167-4baa-8fc3-66d7a52860ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:54.687729 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfff78-4167-4baa-8fc3-66d7a52860ab None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:54.716423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 362/739] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:54.716423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfff78-4167-4baa-8fc3-66d7a52860ab None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:54.762855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde88508-2488-4aac-9114-9accf6318c0b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:54.818448 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:54.837574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673488f-d59f-48ab-9561-6e70d23fe813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:54.838605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673488f-d59f-48ab-9561-6e70d23fe813 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:54.839074 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673488f-d59f-48ab-9561-6e70d23fe813 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:54.839535 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673488f-d59f-48ab-9561-6e70d23fe813 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:54.856456 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673488f-d59f-48ab-9561-6e70d23fe813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:54.865713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-c673488f-d59f-48ab-9561-6e70d23fe813 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:20:54.865713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:54.865713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:54.865713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:54.865713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:54.865713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:54.865713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:54.865713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:54.865713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:54.865713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:54.865713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:54.865713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:54.865713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:54.865713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:54.865713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:54.865713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:54.867849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:54.867849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:54.867849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:54.867849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:54.867849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:54.867849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:54.867849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:54.867849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:54.867849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:54.867849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:54.867849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:54.867849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:54.867849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:54.867849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:54.867849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:54.867849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:54.870321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:54.870321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:54.870321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:54.870321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:54.870321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:54.870321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:54.870321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:54.870321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:54.870321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:54.870321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:54.870321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:20:54.870321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:54.870321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 378/740] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:20:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:54.871929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2291ebb-4989-41b2-8169-ba9b3c524988 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:54.871929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2291ebb-4989-41b2-8169-ba9b3c524988 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:54.871929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2291ebb-4989-41b2-8169-ba9b3c524988 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:54.874187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 379/741] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:54.875050 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2291ebb-4989-41b2-8169-ba9b3c524988 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:54.908920 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfff78-4167-4baa-8fc3-66d7a52860ab None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:54.955017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527be093-ace4-48aa-976e-cf00d989ceed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:54.955964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527be093-ace4-48aa-976e-cf00d989ceed None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:54.956410 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527be093-ace4-48aa-976e-cf00d989ceed None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:54.956852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527be093-ace4-48aa-976e-cf00d989ceed None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:54.958117 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527be093-ace4-48aa-976e-cf00d989ceed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:54.958595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527be093-ace4-48aa-976e-cf00d989ceed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:54.960967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527be093-ace4-48aa-976e-cf00d989ceed None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:54.994352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 363/742] 10.222.0.83 () {56 vars in 1135 bytes} [Mon Aug 1 15:20:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:54.994352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527be093-ace4-48aa-976e-cf00d989ceed None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:55.014434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2291ebb-4989-41b2-8169-ba9b3c524988 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:55.020221 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:55.066087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8f9e0-312c-4f59-9aa2-46e75865405d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:55.066852 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8f9e0-312c-4f59-9aa2-46e75865405d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:55.067145 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8f9e0-312c-4f59-9aa2-46e75865405d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:55.067423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8f9e0-312c-4f59-9aa2-46e75865405d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:55.075395 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8f9e0-312c-4f59-9aa2-46e75865405d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:55.076934 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8f9e0-312c-4f59-9aa2-46e75865405d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:55.089056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-12a8f9e0-312c-4f59-9aa2-46e75865405d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:55.089056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:55.089056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:55.089056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:55.089056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:55.089056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:55.089056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:55.089056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:55.089056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:55.089056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:55.089056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:55.089056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:55.089056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:20:55.089056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:20:55.089056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:20:55.090899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:20:55.090899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:55.090899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:55.090899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:55.090899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:55.090899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:55.090899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:55.090899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:55.090899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:55.090899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:55.090899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:55.090899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:55.090899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:55.090899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:55.090899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:55.090899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:55.090899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:55.092833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:20:55.092833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:20:55.092833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:20:55.092833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:20:55.092833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:20:55.092833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:20:55.092833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:55.092833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:55.092833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 380/743] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:55.092833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa121c-b662-4f49-80cc-cae04551adc1 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:55.092833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa121c-b662-4f49-80cc-cae04551adc1 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:55.094220 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa121c-b662-4f49-80cc-cae04551adc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:55.113331 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-22aa121c-b662-4f49-80cc-cae04551adc1 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:55.114965 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-22aa121c-b662-4f49-80cc-cae04551adc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:55.152997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527be093-ace4-48aa-976e-cf00d989ceed None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:55.163917 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:55.200338 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 381/744] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:55.202630 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be359b77-9b14-4f0c-9c70-606569f1dd04 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:55.202942 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a913f-6aef-4511-892b-4eaa45692c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:55.203129 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be359b77-9b14-4f0c-9c70-606569f1dd04 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:55.203572 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a913f-6aef-4511-892b-4eaa45692c25 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:55.203726 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be359b77-9b14-4f0c-9c70-606569f1dd04 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:55.203850 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a913f-6aef-4511-892b-4eaa45692c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:55.204108 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a913f-6aef-4511-892b-4eaa45692c25 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:55.204855 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a913f-6aef-4511-892b-4eaa45692c25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:55.205160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a913f-6aef-4511-892b-4eaa45692c25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:55.206197 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 382/745] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:55.207483 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be359b77-9b14-4f0c-9c70-606569f1dd04 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:55.211111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a913f-6aef-4511-892b-4eaa45692c25 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:55.231820 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 364/746] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:20:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:55.233671 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a913f-6aef-4511-892b-4eaa45692c25 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:55.341372 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:55.355773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be359b77-9b14-4f0c-9c70-606569f1dd04 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:55.367516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a913f-6aef-4511-892b-4eaa45692c25 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:55.393003 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059387c-f69d-4774-9a61-ec03eadfa7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:55.393824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059387c-f69d-4774-9a61-ec03eadfa7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:55.394199 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059387c-f69d-4774-9a61-ec03eadfa7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:55.394484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059387c-f69d-4774-9a61-ec03eadfa7b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:55.417021 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66009f-f22c-45a9-adff-28bff8c4acf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:55.417964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66009f-f22c-45a9-adff-28bff8c4acf5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:55.418411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66009f-f22c-45a9-adff-28bff8c4acf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:55.418832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66009f-f22c-45a9-adff-28bff8c4acf5 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:55.420045 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66009f-f22c-45a9-adff-28bff8c4acf5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:55.420553 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66009f-f22c-45a9-adff-28bff8c4acf5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:55.422909 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66009f-f22c-45a9-adff-28bff8c4acf5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:55.442237 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 365/747] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:20:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:55.444094 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88443e8e-1c0c-4cc5-b222-b5b6c2da977e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:55.444526 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88443e8e-1c0c-4cc5-b222-b5b6c2da977e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:55.444942 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88443e8e-1c0c-4cc5-b222-b5b6c2da977e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:55.450739 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:55.468436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-88443e8e-1c0c-4cc5-b222-b5b6c2da977e None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:55.469667 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-88443e8e-1c0c-4cc5-b222-b5b6c2da977e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:55.492665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059387c-f69d-4774-9a61-ec03eadfa7b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:55.495430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059387c-f69d-4774-9a61-ec03eadfa7b8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:55.651013 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 383/748] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:20:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:20:55.652176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059387c-f69d-4774-9a61-ec03eadfa7b8 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:55.699558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 366/749] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:20:55.701079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88443e8e-1c0c-4cc5-b222-b5b6c2da977e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:55.802518 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:55.841365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88443e8e-1c0c-4cc5-b222-b5b6c2da977e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:55.879711 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33cd62-50b5-4595-bc3b-30cffd586783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:55.880051 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33cd62-50b5-4595-bc3b-30cffd586783 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:55.880223 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33cd62-50b5-4595-bc3b-30cffd586783 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:55.880546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33cd62-50b5-4595-bc3b-30cffd586783 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:55.895360 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33cd62-50b5-4595-bc3b-30cffd586783 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:55.898118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33cd62-50b5-4595-bc3b-30cffd586783 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:55.900834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:55.923630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-5e33cd62-50b5-4595-bc3b-30cffd586783 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:55.923630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:55.923630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:55.923630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:55.923630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:55.923630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:55.923630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:55.923630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:55.923630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:55.923630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:55.923630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:55.923630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:55.923630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:55.923630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:55.923630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:55.924971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:55.924971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:55.924971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:55.924971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:55.924971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:55.924971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:55.924971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:55.924971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:55.924971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:55.924971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:55.924971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:55.924971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:55.924971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:55.924971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:55.924971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:55.924971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:55.924971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:55.926447 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:55.926447 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:55.926447 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:55.926447 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:55.926447 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 367/750] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:55.926447 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33cd62-50b5-4595-bc3b-30cffd586783 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:55.947089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059387c-f69d-4774-9a61-ec03eadfa7b8 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:55.987227 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e5b4d-1433-490d-b574-bd28ce0b34c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:55.987973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e5b4d-1433-490d-b574-bd28ce0b34c1 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:55.988368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e5b4d-1433-490d-b574-bd28ce0b34c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:55.988768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e5b4d-1433-490d-b574-bd28ce0b34c1 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:55.989679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e5b4d-1433-490d-b574-bd28ce0b34c1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:55.991258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e5b4d-1433-490d-b574-bd28ce0b34c1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:56.013280 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 384/751] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Aug 1 15:20:55 2022] POST /identity/v3/services => generated 251 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:20:56.014872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e5b4d-1433-490d-b574-bd28ce0b34c1 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:56.095693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33cd62-50b5-4595-bc3b-30cffd586783 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:56.138847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:56.170181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a8b3d-235b-4f06-bf49-bb1375c95cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:56.171206 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a8b3d-235b-4f06-bf49-bb1375c95cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:56.171615 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a8b3d-235b-4f06-bf49-bb1375c95cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:56.172042 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a8b3d-235b-4f06-bf49-bb1375c95cc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:56.173406 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a8b3d-235b-4f06-bf49-bb1375c95cc6 None admin] RBAC: Adding query filter params (role.id=091d39715077464daa4cf890db55e6b9, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=f36c6987ef774a1684c55f885e2a4f67) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:56.173990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a8b3d-235b-4f06-bf49-bb1375c95cc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:56.175697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a8b3d-235b-4f06-bf49-bb1375c95cc6 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:56.185650 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 368/752] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:20:55 2022] GET /identity/v3/role_assignments?user.id=f36c6987ef774a1684c55f885e2a4f67&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=091d39715077464daa4cf890db55e6b9 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:56.187230 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a8b3d-235b-4f06-bf49-bb1375c95cc6 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:56.229003 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e5b4d-1433-490d-b574-bd28ce0b34c1 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:56.306705 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8fd1f-cbac-4de0-94aa-6f0c9de53655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:56.307696 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8fd1f-cbac-4de0-94aa-6f0c9de53655 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:56.308193 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8fd1f-cbac-4de0-94aa-6f0c9de53655 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:56.308611 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8fd1f-cbac-4de0-94aa-6f0c9de53655 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:56.311914 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:56.335557 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8fd1f-cbac-4de0-94aa-6f0c9de53655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:56.339382 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8fd1f-cbac-4de0-94aa-6f0c9de53655 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:56.359397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-37b8fd1f-cbac-4de0-94aa-6f0c9de53655 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:56.359397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:56.359397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:56.359397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:56.359397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:56.359397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:56.359397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:56.359397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:56.359397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:56.359397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:56.359397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:56.359397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:56.359397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:56.359397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:56.359397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:56.361058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:56.361058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:56.361058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:56.361058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:56.361058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:56.361058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:56.361058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:56.361058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:56.361058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:56.361058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:56.361058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:56.361058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:56.361058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:56.361058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:56.361058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:56.361058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:56.361058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:56.362894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:56.362894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:56.362894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:56.362894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:56.362894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 385/753] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:56.362894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8fd1f-cbac-4de0-94aa-6f0c9de53655 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:56.385171 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a8b3d-235b-4f06-bf49-bb1375c95cc6 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:56.460443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60dce2-ccb7-4d93-a748-44a2ce43d220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:56.462204 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60dce2-ccb7-4d93-a748-44a2ce43d220 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:56.463007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60dce2-ccb7-4d93-a748-44a2ce43d220 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:56.463768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60dce2-ccb7-4d93-a748-44a2ce43d220 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:56.465426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60dce2-ccb7-4d93-a748-44a2ce43d220 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:56.468379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60dce2-ccb7-4d93-a748-44a2ce43d220 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:56.481519 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8fd1f-cbac-4de0-94aa-6f0c9de53655 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:56.506285 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 369/754] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Aug 1 15:20:56 2022] POST /identity/v3/services => generated 240 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:20:56.509196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60dce2-ccb7-4d93-a748-44a2ce43d220 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:56.534997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c3e7a-0589-4393-a83c-2e6fa652c9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:56.535633 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c3e7a-0589-4393-a83c-2e6fa652c9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:56.535911 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c3e7a-0589-4393-a83c-2e6fa652c9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:56.536174 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c3e7a-0589-4393-a83c-2e6fa652c9d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:56.536873 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334c3e7a-0589-4393-a83c-2e6fa652c9d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:56.537171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334c3e7a-0589-4393-a83c-2e6fa652c9d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:56.538597 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334c3e7a-0589-4393-a83c-2e6fa652c9d2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:56.574495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 386/755] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:56.582948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c3e7a-0589-4393-a83c-2e6fa652c9d2 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:56.610472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:56.686691 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:56.740000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60dce2-ccb7-4d93-a748-44a2ce43d220 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:56.814859 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c3e7a-0589-4393-a83c-2e6fa652c9d2 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:56.820984 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc872b2-5560-4b69-9d0f-0827ec7e8f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:56.821621 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc872b2-5560-4b69-9d0f-0827ec7e8f6d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:56.821927 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc872b2-5560-4b69-9d0f-0827ec7e8f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:56.822206 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc872b2-5560-4b69-9d0f-0827ec7e8f6d None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:56.822899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc872b2-5560-4b69-9d0f-0827ec7e8f6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:56.823204 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc872b2-5560-4b69-9d0f-0827ec7e8f6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:56.824644 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc872b2-5560-4b69-9d0f-0827ec7e8f6d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:56.846893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 370/756] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:56.853982 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc872b2-5560-4b69-9d0f-0827ec7e8f6d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:56.873591 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5de1b5-8ab8-4073-a54e-2715a7908a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:56.874357 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5de1b5-8ab8-4073-a54e-2715a7908a0b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:56.874642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5de1b5-8ab8-4073-a54e-2715a7908a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:56.874982 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5de1b5-8ab8-4073-a54e-2715a7908a0b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:56.884116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5de1b5-8ab8-4073-a54e-2715a7908a0b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:56.885809 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5de1b5-8ab8-4073-a54e-2715a7908a0b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:56.902783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-8a5de1b5-8ab8-4073-a54e-2715a7908a0b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:20:56.902783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:56.902783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:56.902783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:56.902783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:56.902783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:56.902783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:56.902783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:56.902783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:56.902783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:56.902783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:56.902783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:56.902783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:20:56.902783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:20:56.902783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:20:56.902783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:20:56.903882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:56.903882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:56.903882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:20:56.903882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:56.903882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:20:56.903882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:20:56.903882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:56.903882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:56.903882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:56.903882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:56.903882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:56.903882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:56.903882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:56.903882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:56.903882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:56.903882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:56.903882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:56.903882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:56.904834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:56.904834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:56.904834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:20:56.904834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:20:56.904834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:20:56.904834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:20:56.904834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:20:56.904834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:20:56.904834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:20:56.904834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:56.904834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 387/757] 10.222.0.83 () {56 vars in 1093 bytes} [Mon Aug 1 15:20:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:20:56.910586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5de1b5-8ab8-4073-a54e-2715a7908a0b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:56.987045 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc872b2-5560-4b69-9d0f-0827ec7e8f6d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:56.988996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:57.039252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e69a1-b692-4c1a-a957-38685ba097d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:57.040565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e69a1-b692-4c1a-a957-38685ba097d9 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:57.040882 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e69a1-b692-4c1a-a957-38685ba097d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:57.041163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e69a1-b692-4c1a-a957-38685ba097d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:57.051893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5de1b5-8ab8-4073-a54e-2715a7908a0b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:57.056168 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:57.060906 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e69a1-b692-4c1a-a957-38685ba097d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:57.062450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e69a1-b692-4c1a-a957-38685ba097d9 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:57.087219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-a30e69a1-b692-4c1a-a957-38685ba097d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:57.087219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:57.087219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:57.087219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:57.087219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:57.087219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:57.087219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:57.087219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:57.087219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:57.087219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:57.087219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:57.087219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:57.087219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:20:57.087219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:20:57.087219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:20:57.088198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:20:57.088198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:57.088198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:57.088198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:57.088198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:57.088198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:57.088198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:57.088198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:57.088198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:57.088198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:57.088198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:57.088198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:57.088198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:57.088198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:57.088198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:57.088198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:57.088198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:57.089535 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:20:57.089535 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:20:57.089535 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:20:57.089535 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:20:57.089535 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:20:57.089535 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:20:57.089535 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:57.089535 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:57.089535 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 371/758] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:20:57.096367 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e69a1-b692-4c1a-a957-38685ba097d9 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:57.126519 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f5802-043f-4583-aeaf-db20d2f8ed34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:57.128471 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f5802-043f-4583-aeaf-db20d2f8ed34 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:57.129859 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f5802-043f-4583-aeaf-db20d2f8ed34 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:57.130481 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f5802-043f-4583-aeaf-db20d2f8ed34 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:57.132367 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f5802-043f-4583-aeaf-db20d2f8ed34 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:57.133169 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f5802-043f-4583-aeaf-db20d2f8ed34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:57.137990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f5802-043f-4583-aeaf-db20d2f8ed34 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:57.182185 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 388/759] 10.222.0.83 () {56 vars in 1133 bytes} [Mon Aug 1 15:20:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:57.192731 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f5802-043f-4583-aeaf-db20d2f8ed34 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:57.327461 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e69a1-b692-4c1a-a957-38685ba097d9 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:57.399749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417c1f3-d734-49ff-9143-84c268fd92e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:57.400398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417c1f3-d734-49ff-9143-84c268fd92e4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:57.400681 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417c1f3-d734-49ff-9143-84c268fd92e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:57.400958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417c1f3-d734-49ff-9143-84c268fd92e4 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:57.401768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417c1f3-d734-49ff-9143-84c268fd92e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:57.402127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417c1f3-d734-49ff-9143-84c268fd92e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:57.405855 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417c1f3-d734-49ff-9143-84c268fd92e4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:57.413819 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f5802-043f-4583-aeaf-db20d2f8ed34 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:57.432721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 372/760] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:20:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:57.440941 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417c1f3-d734-49ff-9143-84c268fd92e4 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:57.487892 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ebdd4-238c-4419-9534-bed2dc4f1870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:57.488555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ebdd4-238c-4419-9534-bed2dc4f1870 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:57.488832 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ebdd4-238c-4419-9534-bed2dc4f1870 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:57.490339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ebdd4-238c-4419-9534-bed2dc4f1870 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:57.495187 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:57.512022 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ebdd4-238c-4419-9534-bed2dc4f1870 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:57.515866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ebdd4-238c-4419-9534-bed2dc4f1870 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:57.553101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-f01ebdd4-238c-4419-9534-bed2dc4f1870 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:57.553101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:57.553101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:57.553101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:57.553101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:57.553101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:57.553101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:57.553101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:57.553101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:57.553101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:57.553101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:57.553101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:57.553101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:57.553101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:57.553101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:57.554388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:57.554388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:57.554388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:57.554388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:57.554388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:57.554388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:57.554388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:57.554388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:57.554388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:57.554388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:57.554388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:57.554388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:57.554388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:57.554388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:57.554388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:57.554388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:57.554388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:57.555609 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:57.555609 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:57.555609 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:57.555609 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:57.556451 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 389/761] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:57.564476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ebdd4-238c-4419-9534-bed2dc4f1870 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:57.597080 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:57.660948 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417c1f3-d734-49ff-9143-84c268fd92e4 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:57.719071 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246fe0c-fd68-47c0-8f86-1b1a8dd3eb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:57.719707 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246fe0c-fd68-47c0-8f86-1b1a8dd3eb7c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:57.719979 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246fe0c-fd68-47c0-8f86-1b1a8dd3eb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:57.720259 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246fe0c-fd68-47c0-8f86-1b1a8dd3eb7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:57.721060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246fe0c-fd68-47c0-8f86-1b1a8dd3eb7c None admin] RBAC: Adding query filter params (role.id=7bdfa43dc1d046f7965ffe52b74bc58f, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=6873a3e440bd4b63b0082fc85beef49c) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:57.721443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246fe0c-fd68-47c0-8f86-1b1a8dd3eb7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:57.723198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246fe0c-fd68-47c0-8f86-1b1a8dd3eb7c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:57.730298 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 373/762] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:20:57 2022] GET /identity/v3/role_assignments?user.id=6873a3e440bd4b63b0082fc85beef49c&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=7bdfa43dc1d046f7965ffe52b74bc58f => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:57.737384 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ebdd4-238c-4419-9534-bed2dc4f1870 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:57.780263 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6726ab-6fa6-4ac3-ba82-0776847baffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:57.781095 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6726ab-6fa6-4ac3-ba82-0776847baffe None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:57.781385 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6726ab-6fa6-4ac3-ba82-0776847baffe None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:57.781665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6726ab-6fa6-4ac3-ba82-0776847baffe None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:57.782412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6726ab-6fa6-4ac3-ba82-0776847baffe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:57.782722 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6726ab-6fa6-4ac3-ba82-0776847baffe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:57.784137 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6726ab-6fa6-4ac3-ba82-0776847baffe None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:57.810515 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 390/763] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:57.816773 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246fe0c-fd68-47c0-8f86-1b1a8dd3eb7c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:57.943090 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:57.963193 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246fe0c-fd68-47c0-8f86-1b1a8dd3eb7c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:58.016485 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc93ea-410d-47c4-bb4a-a82a0ef42ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:58.017476 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc93ea-410d-47c4-bb4a-a82a0ef42ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:58.017913 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc93ea-410d-47c4-bb4a-a82a0ef42ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:58.018344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc93ea-410d-47c4-bb4a-a82a0ef42ca7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:58.032257 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc93ea-410d-47c4-bb4a-a82a0ef42ca7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:58.034690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc93ea-410d-47c4-bb4a-a82a0ef42ca7 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:58.061842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-4ccc93ea-410d-47c4-bb4a-a82a0ef42ca7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:58.061842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:58.061842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:58.061842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:58.061842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:58.061842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:58.061842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:58.061842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:58.061842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:58.061842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:58.061842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:58.061842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:58.061842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:20:58.061842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:20:58.061842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:20:58.062799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:20:58.062799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:58.062799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:58.062799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:58.062799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:58.062799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:58.062799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:58.062799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:58.062799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:58.062799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:58.062799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:58.062799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:58.062799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:58.062799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:58.062799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:58.062799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:58.062799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:58.063701 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:20:58.063701 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:20:58.063701 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:20:58.063701 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:20:58.063701 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:20:58.063701 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:20:58.063701 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:20:58.063701 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:58.065135 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 374/764] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:20:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:58.077616 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6726ab-6fa6-4ac3-ba82-0776847baffe None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:58.213424 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:58.272482 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6726ab-6fa6-4ac3-ba82-0776847baffe None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:58.326462 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c1be7-e54f-4f44-a479-d5abfd0efe0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:58.327407 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c1be7-e54f-4f44-a479-d5abfd0efe0c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:58.327904 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c1be7-e54f-4f44-a479-d5abfd0efe0c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:58.328326 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c1be7-e54f-4f44-a479-d5abfd0efe0c None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:58.329514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c1be7-e54f-4f44-a479-d5abfd0efe0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:58.330012 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c1be7-e54f-4f44-a479-d5abfd0efe0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:58.332306 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c1be7-e54f-4f44-a479-d5abfd0efe0c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:58.355649 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 391/765] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:20:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:58.361410 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc93ea-410d-47c4-bb4a-a82a0ef42ca7 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:58.528659 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc93ea-410d-47c4-bb4a-a82a0ef42ca7 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:58.584051 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947aa92-2031-45ed-aac1-9ca21f64552d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:58.585116 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947aa92-2031-45ed-aac1-9ca21f64552d None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:58.585563 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947aa92-2031-45ed-aac1-9ca21f64552d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:58.586082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947aa92-2031-45ed-aac1-9ca21f64552d None admin] PATH_INFO: `/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/25dc41ffc41c47fdbeec06110d04610c/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:58.589121 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947aa92-2031-45ed-aac1-9ca21f64552d None admin] RBAC: Authorizing `identity:create_grant(project_id=1f4e27cc50bd4f9f8ced4cc544ab3064, user_id=25dc41ffc41c47fdbeec06110d04610c, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:58.591487 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947aa92-2031-45ed-aac1-9ca21f64552d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:58.618627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 375/766] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:20:58 2022] PUT /identity/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/25dc41ffc41c47fdbeec06110d04610c/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:20:58.753975 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5094ef-5635-4318-9804-623ee9bb9a8c None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:58.754486 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5094ef-5635-4318-9804-623ee9bb9a8c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:58.754593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5094ef-5635-4318-9804-623ee9bb9a8c None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:58.756146 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 392/767] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:20:58.761847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb865b-61fa-4e76-b32d-f2c4a4115eb4 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:58.762190 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb865b-61fa-4e76-b32d-f2c4a4115eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:58.762459 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb865b-61fa-4e76-b32d-f2c4a4115eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:58.779921 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:58.793659 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-87cb865b-61fa-4e76-b32d-f2c4a4115eb4 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:58.794911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-87cb865b-61fa-4e76-b32d-f2c4a4115eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:58.904829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 376/768] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:20:58.937865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5094ef-5635-4318-9804-623ee9bb9a8c None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:59.131283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5094ef-5635-4318-9804-623ee9bb9a8c None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:59.174885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:59.186897 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4662a13-212a-4516-8348-f83b42706b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:59.187860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4662a13-212a-4516-8348-f83b42706b00 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:59.188291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4662a13-212a-4516-8348-f83b42706b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:59.188707 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4662a13-212a-4516-8348-f83b42706b00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:59.211306 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4662a13-212a-4516-8348-f83b42706b00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:20:59.220590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-a4662a13-212a-4516-8348-f83b42706b00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:59.220590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:59.220590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:59.220590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:59.220590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:59.220590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:59.220590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:59.220590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:59.220590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:59.220590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:59.220590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:59.220590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:59.220590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:20:59.220590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:20:59.220590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:20:59.220590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:20:59.222316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:20:59.222316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:20:59.222316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:20:59.222316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:20:59.222316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:59.222316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:59.222316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:59.222316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:59.222316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:59.222316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:59.222316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:59.222316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:59.222316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:59.222316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:59.222316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:59.222316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:59.225171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:59.225171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:59.225171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:59.225171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:59.225171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:20:59.225171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:20:59.225171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:20:59.225171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:20:59.225171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:20:59.225171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:20:59.225171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:20:59.225171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:59.225171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 393/769] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:20:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:59.232052 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb865b-61fa-4e76-b32d-f2c4a4115eb4 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:59.303199 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c95c0-c957-4cf8-892a-010eccee463d None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:59.303458 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c95c0-c957-4cf8-892a-010eccee463d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:59.303794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c95c0-c957-4cf8-892a-010eccee463d None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:59.306072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 394/770] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:20:59.320725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84359c0-8f05-46c3-aeed-d9baa9a27996 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:59.321119 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84359c0-8f05-46c3-aeed-d9baa9a27996 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:59.321300 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84359c0-8f05-46c3-aeed-d9baa9a27996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:59.358954 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c84359c0-8f05-46c3-aeed-d9baa9a27996 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:59.360171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c84359c0-8f05-46c3-aeed-d9baa9a27996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:59.409046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb865b-61fa-4e76-b32d-f2c4a4115eb4 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:59.491111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800d933-772d-4d40-97bb-4bd065bb3641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:59.491766 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800d933-772d-4d40-97bb-4bd065bb3641 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:59.492051 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800d933-772d-4d40-97bb-4bd065bb3641 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:59.492331 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800d933-772d-4d40-97bb-4bd065bb3641 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:59.494401 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1800d933-772d-4d40-97bb-4bd065bb3641 None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:59.494723 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1800d933-772d-4d40-97bb-4bd065bb3641 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:59.496134 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1800d933-772d-4d40-97bb-4bd065bb3641 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:59.508414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:59.516673 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 377/771] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:20:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:59.523656 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800d933-772d-4d40-97bb-4bd065bb3641 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:59.562715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 395/772] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:20:59.591018 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84359c0-8f05-46c3-aeed-d9baa9a27996 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:59.703867 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84359c0-8f05-46c3-aeed-d9baa9a27996 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:59.729629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:59.747194 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f0d00-37f4-4cb3-b8cf-92219313ac45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:59.747849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f0d00-37f4-4cb3-b8cf-92219313ac45 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:59.748132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f0d00-37f4-4cb3-b8cf-92219313ac45 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:59.748408 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f0d00-37f4-4cb3-b8cf-92219313ac45 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:59.749042 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f0d00-37f4-4cb3-b8cf-92219313ac45 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:59.750453 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f0d00-37f4-4cb3-b8cf-92219313ac45 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:59.756879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800d933-772d-4d40-97bb-4bd065bb3641 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:59.771426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-119f0d00-37f4-4cb3-b8cf-92219313ac45 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:20:59.771426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:59.771426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:20:59.771426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:20:59.771426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:20:59.771426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:20:59.771426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:20:59.771426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:20:59.771426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:20:59.771426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:59.771426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:20:59.771426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:59.771426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:59.771426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:59.771426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:59.771426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:59.772893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:59.772893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:59.772893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:59.772893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:59.772893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:59.772893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:59.772893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:59.772893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:20:59.772893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:20:59.772893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:20:59.772893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:20:59.772893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:59.772893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:59.772893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:59.772893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:59.772893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:59.773740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:59.773740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:59.773740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:59.773740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:59.773740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:59.773740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:59.773740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:59.773740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:59.773740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:59.773740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:59.773740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:59.773740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:20:59.773740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:20:59.773740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:20:59.773740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:20:59.773740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 396/773] 10.222.0.83 () {56 vars in 1129 bytes} [Mon Aug 1 15:20:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:20:59.797489 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f0d00-37f4-4cb3-b8cf-92219313ac45 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:59.831385 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f03c6f-b5ad-44a0-b41b-758532a58092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:59.831959 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f03c6f-b5ad-44a0-b41b-758532a58092 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:59.832241 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f03c6f-b5ad-44a0-b41b-758532a58092 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:59.832532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f03c6f-b5ad-44a0-b41b-758532a58092 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:59.850332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f03c6f-b5ad-44a0-b41b-758532a58092 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:59.851802 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f03c6f-b5ad-44a0-b41b-758532a58092 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:59.892710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-b9f03c6f-b5ad-44a0-b41b-758532a58092 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:59.892710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:20:59.892710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:20:59.892710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:20:59.892710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:20:59.892710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:20:59.892710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:20:59.892710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:20:59.892710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:20:59.892710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:20:59.892710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:20:59.892710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:20:59.892710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:20:59.892710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:20:59.892710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:20:59.895433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:20:59.895433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:20:59.895433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:20:59.895433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:20:59.895433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:20:59.895433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:20:59.895433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:20:59.895433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:20:59.895433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:20:59.895433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:20:59.895433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:20:59.895433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:20:59.895433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:20:59.895433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:20:59.895433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:20:59.895433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:20:59.895433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:20:59.898312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:20:59.898312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:20:59.898312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:20:59.898312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:20:59.898312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 378/774] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:20:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:20:59.903804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f03c6f-b5ad-44a0-b41b-758532a58092 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:59.963253 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:59.988932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f0d00-37f4-4cb3-b8cf-92219313ac45 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:00.029986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a52b09-7aa7-48da-8071-c83d1272230d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:00.030632 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a52b09-7aa7-48da-8071-c83d1272230d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:00.030915 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a52b09-7aa7-48da-8071-c83d1272230d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:00.031191 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a52b09-7aa7-48da-8071-c83d1272230d None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:00.031910 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a52b09-7aa7-48da-8071-c83d1272230d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:00.032218 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a52b09-7aa7-48da-8071-c83d1272230d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:00.033624 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a52b09-7aa7-48da-8071-c83d1272230d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:00.038209 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f03c6f-b5ad-44a0-b41b-758532a58092 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:00.040115 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 397/775] 10.222.0.83 () {56 vars in 1124 bytes} [Mon Aug 1 15:20:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:00.041540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb24cf5-da28-4672-baf3-c5a8bd463635 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:00.041835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb24cf5-da28-4672-baf3-c5a8bd463635 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:00.042147 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb24cf5-da28-4672-baf3-c5a8bd463635 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:00.043585 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 398/776] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:00.048429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb24cf5-da28-4672-baf3-c5a8bd463635 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:00.090965 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84587f7-21ca-47a6-96ca-ce12d305d3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:00.091907 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84587f7-21ca-47a6-96ca-ce12d305d3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:00.092373 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84587f7-21ca-47a6-96ca-ce12d305d3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:00.092802 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84587f7-21ca-47a6-96ca-ce12d305d3a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:00.093681 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84587f7-21ca-47a6-96ca-ce12d305d3a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:00.094259 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84587f7-21ca-47a6-96ca-ce12d305d3a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:00.096026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84587f7-21ca-47a6-96ca-ce12d305d3a3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:00.111962 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 379/777] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:20:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:00.113609 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23159652-3545-402c-8911-3b4125cc88a9 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:00.114148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23159652-3545-402c-8911-3b4125cc88a9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:00.114633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23159652-3545-402c-8911-3b4125cc88a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:00.135082 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:00.139924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-23159652-3545-402c-8911-3b4125cc88a9 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:00.141961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-23159652-3545-402c-8911-3b4125cc88a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:00.213048 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb24cf5-da28-4672-baf3-c5a8bd463635 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:00.238245 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 380/778] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:00.239707 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23159652-3545-402c-8911-3b4125cc88a9 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:00.255928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b014318-1636-4379-80b0-58e8273d84aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:00.256245 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b014318-1636-4379-80b0-58e8273d84aa None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:00.256504 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b014318-1636-4379-80b0-58e8273d84aa None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:00.256734 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b014318-1636-4379-80b0-58e8273d84aa None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:00.257367 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b014318-1636-4379-80b0-58e8273d84aa None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:00.257673 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b014318-1636-4379-80b0-58e8273d84aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:00.259524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b014318-1636-4379-80b0-58e8273d84aa None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:00.266462 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 399/779] 10.222.0.83 () {56 vars in 1124 bytes} [Mon Aug 1 15:21:00 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:00.273769 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b014318-1636-4379-80b0-58e8273d84aa None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:00.315235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:00.389830 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23159652-3545-402c-8911-3b4125cc88a9 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:00.405921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b014318-1636-4379-80b0-58e8273d84aa None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:00.419207 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:00.440939 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330d29a-595a-4d73-85c8-f963394f437b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:00.441569 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330d29a-595a-4d73-85c8-f963394f437b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:00.441851 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330d29a-595a-4d73-85c8-f963394f437b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:00.442153 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330d29a-595a-4d73-85c8-f963394f437b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:00.442916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330d29a-595a-4d73-85c8-f963394f437b None admin] RBAC: Adding query filter params (interface=public, service_id=f4740913a5e44330969a961513190feb, region_id=RegionOne) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:00.443221 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330d29a-595a-4d73-85c8-f963394f437b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:00.444588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330d29a-595a-4d73-85c8-f963394f437b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:00.449245 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef44b2-2157-4572-8a2e-4d4c5dc84c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:00.450429 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef44b2-2157-4572-8a2e-4d4c5dc84c77 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:00.450933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef44b2-2157-4572-8a2e-4d4c5dc84c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:00.451478 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef44b2-2157-4572-8a2e-4d4c5dc84c77 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:00.458173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 400/780] 10.222.0.83 () {56 vars in 1251 bytes} [Mon Aug 1 15:21:00 2022] GET /identity/v3/endpoints?service_id=f4740913a5e44330969a961513190feb&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:00.459176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330d29a-595a-4d73-85c8-f963394f437b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:00.465512 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cef44b2-2157-4572-8a2e-4d4c5dc84c77 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:00.468105 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cef44b2-2157-4572-8a2e-4d4c5dc84c77 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:00.490396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-2cef44b2-2157-4572-8a2e-4d4c5dc84c77 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:21:00.490396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:00.490396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:00.490396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:00.490396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:00.490396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:00.490396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:00.490396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:00.490396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:00.490396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:00.490396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:00.490396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:00.490396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:21:00.490396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:21:00.490396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:21:00.490396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:21:00.491508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:00.491508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:00.491508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:21:00.491508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:00.491508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:21:00.491508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:00.491508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:00.491508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:00.491508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:00.491508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:00.491508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:00.491508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:00.491508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:00.491508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:00.491508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:00.491508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:00.491508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:00.491508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:00.492505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:00.492505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:00.492505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:21:00.492505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:21:00.492505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:21:00.492505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:21:00.492505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:21:00.492505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:21:00.492505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:21:00.492505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:00.493075 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 381/781] 10.222.0.83 () {56 vars in 1117 bytes} [Mon Aug 1 15:21:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:00.495081 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef44b2-2157-4572-8a2e-4d4c5dc84c77 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:00.582115 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330d29a-595a-4d73-85c8-f963394f437b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:00.619890 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73082c04-dcc9-4e85-b486-98c04651e73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:00.620552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73082c04-dcc9-4e85-b486-98c04651e73b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:00.620832 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73082c04-dcc9-4e85-b486-98c04651e73b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:00.621096 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73082c04-dcc9-4e85-b486-98c04651e73b None admin] PATH_INFO: `/v3/services/network` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:00.621726 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73082c04-dcc9-4e85-b486-98c04651e73b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:00.623134 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73082c04-dcc9-4e85-b486-98c04651e73b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:00.633036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-73082c04-dcc9-4e85-b486-98c04651e73b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:21:00.633036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:00.633036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:21:00.633036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:21:00.633036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:21:00.633036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:21:00.633036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:21:00.633036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:00.633036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:21:00.633036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:00.633036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:00.633036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:00.633036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:00.633036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:00.633036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:00.633036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:00.633036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:00.636098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:00.636098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:00.636098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:00.636098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:00.636098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:00.636098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:00.636098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:21:00.636098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:21:00.636098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:21:00.636098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:21:00.636098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:00.636098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:00.636098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:00.636098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:00.636098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:00.636098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:00.637475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:00.637475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:00.637475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:00.637475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:00.637475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:00.637475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:00.637475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:00.637475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:00.637475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:00.637475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:00.637475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:21:00.637475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:00.637475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:21:00.637475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:00.637475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 401/782] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:21:00 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:00.638785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73082c04-dcc9-4e85-b486-98c04651e73b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:00.665297 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:00.669203 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef44b2-2157-4572-8a2e-4d4c5dc84c77 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:00.698924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:00.718863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c21783-56be-4cb1-aab1-4e69dff33ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:00.719691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c21783-56be-4cb1-aab1-4e69dff33ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:00.720123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c21783-56be-4cb1-aab1-4e69dff33ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:00.720555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c21783-56be-4cb1-aab1-4e69dff33ab0 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:00.721414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c21783-56be-4cb1-aab1-4e69dff33ab0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:00.723025 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c21783-56be-4cb1-aab1-4e69dff33ab0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:00.737847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 382/783] 10.222.0.83 () {56 vars in 1087 bytes} [Mon Aug 1 15:21:00 2022] GET /identity/v3/services => generated 772 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:00.737847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a630fbf-64d3-4361-ad6e-264ecd4526e3 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:00.737847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a630fbf-64d3-4361-ad6e-264ecd4526e3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:00.737847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a630fbf-64d3-4361-ad6e-264ecd4526e3 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:00.737847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 383/784] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:00.737847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a630fbf-64d3-4361-ad6e-264ecd4526e3 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:00.777756 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73082c04-dcc9-4e85-b486-98c04651e73b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:00.826283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e37072-42f0-41d6-b606-d4b0b64a69fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:00.827347 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e37072-42f0-41d6-b606-d4b0b64a69fb None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:00.827811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e37072-42f0-41d6-b606-d4b0b64a69fb None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:00.828164 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e37072-42f0-41d6-b606-d4b0b64a69fb None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:00.829564 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e37072-42f0-41d6-b606-d4b0b64a69fb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:00.830044 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e37072-42f0-41d6-b606-d4b0b64a69fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:00.831818 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e37072-42f0-41d6-b606-d4b0b64a69fb None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:00.853100 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 402/785] 10.222.0.83 () {56 vars in 1149 bytes} [Mon Aug 1 15:21:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:00.854561 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90fcbb-dcb2-4337-b78a-3d2afff4e882 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:00.854847 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90fcbb-dcb2-4337-b78a-3d2afff4e882 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:00.855211 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90fcbb-dcb2-4337-b78a-3d2afff4e882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:00.878939 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-4e90fcbb-dcb2-4337-b78a-3d2afff4e882 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:00.880347 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4e90fcbb-dcb2-4337-b78a-3d2afff4e882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:00.944336 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a630fbf-64d3-4361-ad6e-264ecd4526e3 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:00.983821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4c307-789a-4d6a-8c34-1305ebedc345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:00.984464 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4c307-789a-4d6a-8c34-1305ebedc345 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:00.984746 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4c307-789a-4d6a-8c34-1305ebedc345 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:00.985018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4c307-789a-4d6a-8c34-1305ebedc345 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:00.985728 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f4c307-789a-4d6a-8c34-1305ebedc345 None admin] RBAC: Adding query filter params (name=network) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:00.986056 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f4c307-789a-4d6a-8c34-1305ebedc345 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:00.988366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f4c307-789a-4d6a-8c34-1305ebedc345 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:01.000988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 384/786] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Aug 1 15:21:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:01.001957 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4c307-789a-4d6a-8c34-1305ebedc345 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:01.016107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 403/787] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:01.017246 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90fcbb-dcb2-4337-b78a-3d2afff4e882 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:01.059974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:01.072558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:01.157065 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4c307-789a-4d6a-8c34-1305ebedc345 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:01.187004 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90fcbb-dcb2-4337-b78a-3d2afff4e882 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:01.218222 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e84112-5e8d-4978-947b-0acf768e7ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:01.219386 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e84112-5e8d-4978-947b-0acf768e7ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:01.220234 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e84112-5e8d-4978-947b-0acf768e7ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:01.221008 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e84112-5e8d-4978-947b-0acf768e7ed8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:01.237176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5040bc1-0a09-42ea-bd1b-6abe56bfd9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:01.238031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5040bc1-0a09-42ea-bd1b-6abe56bfd9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:01.238446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5040bc1-0a09-42ea-bd1b-6abe56bfd9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:01.238889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5040bc1-0a09-42ea-bd1b-6abe56bfd9b2 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:01.240274 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5040bc1-0a09-42ea-bd1b-6abe56bfd9b2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:01.240719 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5040bc1-0a09-42ea-bd1b-6abe56bfd9b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:01.242483 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5040bc1-0a09-42ea-bd1b-6abe56bfd9b2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:01.243163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e84112-5e8d-4978-947b-0acf768e7ed8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:01.248924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e84112-5e8d-4978-947b-0acf768e7ed8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:01.249589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 404/788] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Aug 1 15:21:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:01.250616 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5040bc1-0a09-42ea-bd1b-6abe56bfd9b2 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:01.278618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-74e84112-5e8d-4978-947b-0acf768e7ed8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:01.278618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:01.278618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:01.278618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:01.278618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:01.278618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:01.278618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:01.278618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:01.278618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:01.278618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:01.278618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:01.278618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:01.278618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:01.278618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:01.278618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:01.279708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:01.279708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:01.279708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:01.279708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:01.279708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:01.279708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:01.279708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:01.279708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:01.279708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:01.279708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:01.279708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:01.279708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:01.279708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:01.279708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:01.279708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:01.279708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:01.279708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:01.280791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:01.280791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:01.280791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:01.280791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:01.282913 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 385/789] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:01.284821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3c5e0-0178-42cc-ab5a-efd1d6e40e84 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:01.285332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3c5e0-0178-42cc-ab5a-efd1d6e40e84 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:01.285713 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3c5e0-0178-42cc-ab5a-efd1d6e40e84 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:01.294500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 386/790] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:01.294500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3c5e0-0178-42cc-ab5a-efd1d6e40e84 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:01.391303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:01.396215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:01.401099 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5040bc1-0a09-42ea-bd1b-6abe56bfd9b2 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:01.426268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3c5e0-0178-42cc-ab5a-efd1d6e40e84 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:01.452146 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03083162-0374-4b7b-94af-cf5e93922167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:01.453107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03083162-0374-4b7b-94af-cf5e93922167 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:01.453537 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03083162-0374-4b7b-94af-cf5e93922167 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:01.454011 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03083162-0374-4b7b-94af-cf5e93922167 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:01.467280 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88787de2-3c9f-4f0d-aa20-740172640a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:01.468712 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88787de2-3c9f-4f0d-aa20-740172640a02 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:01.469002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88787de2-3c9f-4f0d-aa20-740172640a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:01.469278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88787de2-3c9f-4f0d-aa20-740172640a02 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:01.470203 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88787de2-3c9f-4f0d-aa20-740172640a02 None admin] RBAC: Adding query filter params (interface=public, service_id=8580bd37bd644f148a2184d764cda6f8, region_id=RegionOne) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:01.470515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88787de2-3c9f-4f0d-aa20-740172640a02 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:01.471950 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88787de2-3c9f-4f0d-aa20-740172640a02 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:01.472324 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03083162-0374-4b7b-94af-cf5e93922167 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:21:01.482487 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 387/791] 10.222.0.83 () {56 vars in 1251 bytes} [Mon Aug 1 15:21:01 2022] GET /identity/v3/endpoints?service_id=8580bd37bd644f148a2184d764cda6f8&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:01.483087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-03083162-0374-4b7b-94af-cf5e93922167 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:21:01.483087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:01.483087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:01.483087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:01.483087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:01.483087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:01.483087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:01.483087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:01.483087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:01.483087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:01.483087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:01.483087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:01.483087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:21:01.483087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:21:01.483087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:21:01.483087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:21:01.484613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:21:01.484613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:21:01.484613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:21:01.484613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:21:01.484613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:01.484613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:01.484613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:01.484613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:01.484613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:01.484613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:01.484613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:01.484613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:01.484613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:01.484613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:01.484613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:01.484613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:01.486744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:01.486744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:01.486744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:01.486744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:01.486744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:21:01.486744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:21:01.486744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:21:01.486744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:21:01.486744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:21:01.486744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:21:01.486744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:21:01.486744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:01.486744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 405/792] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:21:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:01.487959 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5240f7-b085-4837-a937-04bb3dab2e86 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:01.488375 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5240f7-b085-4837-a937-04bb3dab2e86 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:01.488601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5240f7-b085-4837-a937-04bb3dab2e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:01.490386 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88787de2-3c9f-4f0d-aa20-740172640a02 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:01.517901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-de5240f7-b085-4837-a937-04bb3dab2e86 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:01.519828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-de5240f7-b085-4837-a937-04bb3dab2e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:01.635073 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 406/793] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:01.635616 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88787de2-3c9f-4f0d-aa20-740172640a02 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:01.635934 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5240f7-b085-4837-a937-04bb3dab2e86 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:01.684500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ff312-62c7-4d9e-8d09-aeafdc86c10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:01.685259 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ff312-62c7-4d9e-8d09-aeafdc86c10a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:01.685544 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ff312-62c7-4d9e-8d09-aeafdc86c10a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:01.685824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ff312-62c7-4d9e-8d09-aeafdc86c10a None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:01.686570 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ff312-62c7-4d9e-8d09-aeafdc86c10a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:01.686878 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ff312-62c7-4d9e-8d09-aeafdc86c10a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:01.688318 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ff312-62c7-4d9e-8d09-aeafdc86c10a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:01.703202 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 388/794] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:01.704189 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ff312-62c7-4d9e-8d09-aeafdc86c10a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:01.711016 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:01.765964 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:01.787264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5240f7-b085-4837-a937-04bb3dab2e86 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:01.832491 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55791b22-f569-41b0-96b1-726051057f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:01.833173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55791b22-f569-41b0-96b1-726051057f57 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:01.833462 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55791b22-f569-41b0-96b1-726051057f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:01.833733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55791b22-f569-41b0-96b1-726051057f57 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:01.834424 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55791b22-f569-41b0-96b1-726051057f57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:01.835811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55791b22-f569-41b0-96b1-726051057f57 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:01.842364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 407/795] 10.222.0.83 () {56 vars in 1087 bytes} [Mon Aug 1 15:21:01 2022] GET /identity/v3/services => generated 772 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:01.843463 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55791b22-f569-41b0-96b1-726051057f57 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:01.854877 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ff312-62c7-4d9e-8d09-aeafdc86c10a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:01.933941 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f10a47-1a19-4b0b-bde7-cdfd9522966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:01.935035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f10a47-1a19-4b0b-bde7-cdfd9522966b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:01.935316 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f10a47-1a19-4b0b-bde7-cdfd9522966b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:01.935760 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f10a47-1a19-4b0b-bde7-cdfd9522966b None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:01.936589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f10a47-1a19-4b0b-bde7-cdfd9522966b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:01.936896 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f10a47-1a19-4b0b-bde7-cdfd9522966b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:01.938466 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f10a47-1a19-4b0b-bde7-cdfd9522966b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:01.956853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 389/796] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:21:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:01.957859 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f10a47-1a19-4b0b-bde7-cdfd9522966b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:02.009149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55791b22-f569-41b0-96b1-726051057f57 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:02.058402 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a54e5-d819-4b3a-8f8b-d19c3d64983d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:02.059051 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a54e5-d819-4b3a-8f8b-d19c3d64983d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:02.059428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a54e5-d819-4b3a-8f8b-d19c3d64983d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:02.059581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a54e5-d819-4b3a-8f8b-d19c3d64983d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:02.080397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a54e5-d819-4b3a-8f8b-d19c3d64983d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:21:02.087186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-e08a54e5-d819-4b3a-8f8b-d19c3d64983d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:21:02.087186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:02.087186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:02.087186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:02.087186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:02.087186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:02.087186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:02.087186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:02.087186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:02.087186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:02.087186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:02.087186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:02.087186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:21:02.087186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:21:02.087186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:21:02.087186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:21:02.088446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:21:02.088446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:21:02.088446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:21:02.088446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:21:02.088446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:02.088446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:02.088446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:02.088446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:02.088446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:02.088446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:02.088446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:02.088446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:02.088446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:02.088446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:02.088446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:02.088446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:02.089329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:02.089329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:02.089329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:02.089329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:02.089329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:21:02.089329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:21:02.089329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:21:02.089329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:21:02.089329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:21:02.089329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:21:02.089329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:21:02.089329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:02.089329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 408/797] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:21:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:02.090146 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a54e5-d819-4b3a-8f8b-d19c3d64983d None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:02.101275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:02.110858 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:02.153228 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f10a47-1a19-4b0b-bde7-cdfd9522966b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:02.198618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b8bcc-458e-4cd2-8e4f-6d3fa094ad02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:02.199812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b8bcc-458e-4cd2-8e4f-6d3fa094ad02 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:02.200289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b8bcc-458e-4cd2-8e4f-6d3fa094ad02 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:02.200838 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b8bcc-458e-4cd2-8e4f-6d3fa094ad02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:02.214266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901b8bcc-458e-4cd2-8e4f-6d3fa094ad02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:02.216053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901b8bcc-458e-4cd2-8e4f-6d3fa094ad02 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:02.232789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-901b8bcc-458e-4cd2-8e4f-6d3fa094ad02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:02.232789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:02.232789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:02.232789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:02.232789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:02.232789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:02.232789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:02.232789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:02.232789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:02.232789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:02.232789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:02.232789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:02.232789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:21:02.232789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:21:02.232789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:21:02.233770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:21:02.233770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:02.233770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:02.233770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:02.233770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:02.233770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:02.233770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:02.233770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:02.233770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:02.233770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:02.233770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:02.233770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:02.233770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:02.233770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:02.233770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:02.233770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:02.233770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:02.234710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:21:02.234710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:21:02.234710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:21:02.234710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:21:02.234710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:21:02.234710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:21:02.234710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:02.234710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:02.238227 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 390/798] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:21:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:02.239331 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b8bcc-458e-4cd2-8e4f-6d3fa094ad02 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:02.273156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a54e5-d819-4b3a-8f8b-d19c3d64983d None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:02.334419 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcab8d-b6f4-4bc0-a625-f5af5db8bddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:02.335510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcab8d-b6f4-4bc0-a625-f5af5db8bddf None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:02.337292 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcab8d-b6f4-4bc0-a625-f5af5db8bddf None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:02.337850 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcab8d-b6f4-4bc0-a625-f5af5db8bddf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:02.352532 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcab8d-b6f4-4bc0-a625-f5af5db8bddf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:02.355145 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcab8d-b6f4-4bc0-a625-f5af5db8bddf None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:02.375393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-2cbcab8d-b6f4-4bc0-a625-f5af5db8bddf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:02.375393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:02.375393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:02.375393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:02.375393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:02.375393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:02.375393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:02.375393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:02.375393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:02.375393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:02.375393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:02.375393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:02.375393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:02.375393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:02.375393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:02.376948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:02.376948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:02.376948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:02.376948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:02.376948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:02.376948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:02.376948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:02.376948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:02.376948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:02.376948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:02.376948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:02.376948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:02.376948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:02.376948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:02.376948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:02.376948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:02.376948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:02.378004 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:02.378004 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:02.378004 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:02.378004 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:02.380320 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 409/799] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:02.381939 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcab8d-b6f4-4bc0-a625-f5af5db8bddf None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:02.403731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b8bcc-458e-4cd2-8e4f-6d3fa094ad02 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:02.449739 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:02.464323 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85400fdd-2263-4a60-b699-11a019573b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:02.465409 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85400fdd-2263-4a60-b699-11a019573b2f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:02.466246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85400fdd-2263-4a60-b699-11a019573b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:02.466759 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85400fdd-2263-4a60-b699-11a019573b2f None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:02.468667 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85400fdd-2263-4a60-b699-11a019573b2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:02.470424 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85400fdd-2263-4a60-b699-11a019573b2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:02.474221 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85400fdd-2263-4a60-b699-11a019573b2f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:02.479141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:02.489347 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 391/800] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:21:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:02.491010 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85400fdd-2263-4a60-b699-11a019573b2f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:02.554523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcab8d-b6f4-4bc0-a625-f5af5db8bddf None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:02.610916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81490073-e9fa-4837-8b95-a890df4a2db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:02.611880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81490073-e9fa-4837-8b95-a890df4a2db1 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:02.612313 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81490073-e9fa-4837-8b95-a890df4a2db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:02.612738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81490073-e9fa-4837-8b95-a890df4a2db1 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:02.613968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81490073-e9fa-4837-8b95-a890df4a2db1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:02.614455 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81490073-e9fa-4837-8b95-a890df4a2db1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:02.617155 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81490073-e9fa-4837-8b95-a890df4a2db1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:02.623172 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85400fdd-2263-4a60-b699-11a019573b2f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:02.646041 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 410/801] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:21:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:02.647715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81490073-e9fa-4837-8b95-a890df4a2db1 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:02.662583 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eca770-0ec7-424b-94ff-48adf53db3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:02.664168 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eca770-0ec7-424b-94ff-48adf53db3ce None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:02.664721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eca770-0ec7-424b-94ff-48adf53db3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:02.665209 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eca770-0ec7-424b-94ff-48adf53db3ce None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:02.666294 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eca770-0ec7-424b-94ff-48adf53db3ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:02.666819 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eca770-0ec7-424b-94ff-48adf53db3ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:02.668945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eca770-0ec7-424b-94ff-48adf53db3ce None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:02.687910 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 392/802] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:02.689503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eca770-0ec7-424b-94ff-48adf53db3ce None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:02.793574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:02.830889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81490073-e9fa-4837-8b95-a890df4a2db1 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:02.869768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:02.893588 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eca770-0ec7-424b-94ff-48adf53db3ce None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:02.896257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dde4a8-db55-4661-b9df-8f4ffaf57fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:02.898507 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dde4a8-db55-4661-b9df-8f4ffaf57fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:02.898680 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dde4a8-db55-4661-b9df-8f4ffaf57fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:02.899132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dde4a8-db55-4661-b9df-8f4ffaf57fa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:02.920230 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dde4a8-db55-4661-b9df-8f4ffaf57fa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:02.927937 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dde4a8-db55-4661-b9df-8f4ffaf57fa2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:02.958475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-b4dde4a8-db55-4661-b9df-8f4ffaf57fa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:02.958475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:02.958475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:02.958475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:02.958475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:02.958475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:02.958475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:02.958475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:02.958475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:02.958475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:02.958475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:02.958475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:02.958475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:02.958475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:02.958475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:02.960235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:02.960235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:02.960235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:02.960235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:02.960235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:02.960235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:02.960235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:02.960235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:02.960235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:02.960235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:02.960235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:02.960235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:02.960235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:02.960235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:02.960235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:02.960235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:02.960235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:02.962045 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:02.962045 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:02.962045 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:02.962045 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:02.965986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 411/803] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:02.967692 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dde4a8-db55-4661-b9df-8f4ffaf57fa2 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:02.968866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42a78d-3fd2-4f8c-af32-c706423ef474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:02.969688 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42a78d-3fd2-4f8c-af32-c706423ef474 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:02.970131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42a78d-3fd2-4f8c-af32-c706423ef474 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:02.970547 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42a78d-3fd2-4f8c-af32-c706423ef474 None admin] PATH_INFO: `/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/f36c6987ef774a1684c55f885e2a4f67/roles/091d39715077464daa4cf890db55e6b9` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:02.998827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42a78d-3fd2-4f8c-af32-c706423ef474 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f4e27cc50bd4f9f8ced4cc544ab3064, user_id=f36c6987ef774a1684c55f885e2a4f67, role_id=091d39715077464daa4cf890db55e6b9)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:03.000610 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42a78d-3fd2-4f8c-af32-c706423ef474 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:03.025018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 393/804] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:21:02 2022] PUT /identity/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/f36c6987ef774a1684c55f885e2a4f67/roles/091d39715077464daa4cf890db55e6b9 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:21:03.026507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42a78d-3fd2-4f8c-af32-c706423ef474 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:03.153380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dde4a8-db55-4661-b9df-8f4ffaf57fa2 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:03.200475 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:03.208023 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42a78d-3fd2-4f8c-af32-c706423ef474 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:03.234983 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c0315-b3fa-43a7-8ed4-8bece85d4f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:03.235650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c0315-b3fa-43a7-8ed4-8bece85d4f09 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:03.235964 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c0315-b3fa-43a7-8ed4-8bece85d4f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:03.236482 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c0315-b3fa-43a7-8ed4-8bece85d4f09 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:03.249161 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c0315-b3fa-43a7-8ed4-8bece85d4f09 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:03.251003 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c0315-b3fa-43a7-8ed4-8bece85d4f09 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:03.258016 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4d083-276a-4619-b285-7acc896e81b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:03.259013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4d083-276a-4619-b285-7acc896e81b0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:03.259396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4d083-276a-4619-b285-7acc896e81b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:03.259849 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4d083-276a-4619-b285-7acc896e81b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:03.260803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4d083-276a-4619-b285-7acc896e81b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:03.261268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4d083-276a-4619-b285-7acc896e81b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:03.262859 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4d083-276a-4619-b285-7acc896e81b0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:03.266845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-459c0315-b3fa-43a7-8ed4-8bece85d4f09 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:21:03.266845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:03.266845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:03.266845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:03.266845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:03.266845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:03.266845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:03.266845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:03.266845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:03.266845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:03.266845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:03.266845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:03.266845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:21:03.266845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:21:03.266845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:21:03.266845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:21:03.268520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:03.268520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:03.268520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:21:03.268520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:03.268520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:21:03.268520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:03.268520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:03.268520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:03.268520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:03.268520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:03.268520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:03.268520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:03.268520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:03.268520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:03.268520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:03.268520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:03.268520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:03.268520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:03.270229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:03.270229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:03.270229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:21:03.270229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:21:03.270229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:21:03.270229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:21:03.270229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:21:03.270229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:21:03.270229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:21:03.270229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:03.270229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 412/805] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:21:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:03.270229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa3834-7515-4f29-b99d-bf669ce7d27d None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:03.270229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa3834-7515-4f29-b99d-bf669ce7d27d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:03.271541 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa3834-7515-4f29-b99d-bf669ce7d27d None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:03.276181 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 413/806] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:03.276181 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa3834-7515-4f29-b99d-bf669ce7d27d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:03.290266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 394/807] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:03.291650 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb8bf5-2ff9-491f-8e01-31f769ab0aca None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:03.291942 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb8bf5-2ff9-491f-8e01-31f769ab0aca None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:03.292231 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb8bf5-2ff9-491f-8e01-31f769ab0aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:03.314292 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-8bbb8bf5-2ff9-491f-8e01-31f769ab0aca None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:03.315576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8bbb8bf5-2ff9-491f-8e01-31f769ab0aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:03.336025 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:03.421365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa3834-7515-4f29-b99d-bf669ce7d27d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:03.422586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 395/808] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:03.424040 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb8bf5-2ff9-491f-8e01-31f769ab0aca None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:03.458503 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d94ae-4d8c-4333-8f4f-00c695135844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:03.459295 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d94ae-4d8c-4333-8f4f-00c695135844 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:03.459579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d94ae-4d8c-4333-8f4f-00c695135844 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:03.459851 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d94ae-4d8c-4333-8f4f-00c695135844 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:03.460776 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d94ae-4d8c-4333-8f4f-00c695135844 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:03.461091 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d94ae-4d8c-4333-8f4f-00c695135844 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:03.463659 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d94ae-4d8c-4333-8f4f-00c695135844 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:03.493727 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 414/809] 10.222.0.83 () {56 vars in 1135 bytes} [Mon Aug 1 15:21:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:03.495036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d94ae-4d8c-4333-8f4f-00c695135844 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:03.603093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:03.625174 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb8bf5-2ff9-491f-8e01-31f769ab0aca None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:03.642342 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:03.650662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d94ae-4d8c-4333-8f4f-00c695135844 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:03.684638 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1510e-dc95-4f73-b822-c698fc1a6674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:03.686326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1510e-dc95-4f73-b822-c698fc1a6674 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:03.687057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1510e-dc95-4f73-b822-c698fc1a6674 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:03.687849 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1510e-dc95-4f73-b822-c698fc1a6674 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:03.689540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3260f-1af8-40fb-977a-ddca02527a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:03.690230 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3260f-1af8-40fb-977a-ddca02527a28 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:03.690509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3260f-1af8-40fb-977a-ddca02527a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:03.690786 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3260f-1af8-40fb-977a-ddca02527a28 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:03.691416 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd3260f-1af8-40fb-977a-ddca02527a28 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:03.692914 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd3260f-1af8-40fb-977a-ddca02527a28 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:03.702948 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1510e-dc95-4f73-b822-c698fc1a6674 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:03.703693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-9dd3260f-1af8-40fb-977a-ddca02527a28 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:21:03.703693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:03.703693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:21:03.703693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:21:03.703693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:21:03.703693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:21:03.703693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:21:03.703693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:03.703693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:21:03.703693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:03.703693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:03.703693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:03.703693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:03.703693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:03.703693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:03.703693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:03.705437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:03.705437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:03.705437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:03.705437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:03.705437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:03.705437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:03.705437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:03.705437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:21:03.705437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:21:03.705437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:21:03.705437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:21:03.705437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:03.705437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:03.705437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:03.705437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:03.705437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:03.707388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:03.707388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:03.707388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:03.707388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:03.707388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:03.707388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:03.707388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:03.707388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:03.707388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:03.707388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:03.707388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:03.707388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:21:03.707388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:03.707388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:21:03.707388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:03.707388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 415/810] 10.222.0.83 () {56 vars in 1129 bytes} [Mon Aug 1 15:21:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:03.710369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3260f-1af8-40fb-977a-ddca02527a28 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:03.710574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1510e-dc95-4f73-b822-c698fc1a6674 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:03.728627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-fbb1510e-dc95-4f73-b822-c698fc1a6674 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:21:03.728627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:03.728627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:03.728627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:03.728627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:03.728627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:03.728627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:03.728627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:03.728627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:03.728627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:03.728627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:03.728627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:03.728627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:21:03.728627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:21:03.728627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:21:03.728627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:21:03.731961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:03.731961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:03.731961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:21:03.731961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:03.731961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:21:03.731961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:03.731961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:03.731961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:03.731961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:03.731961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:03.731961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:03.731961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:03.731961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:03.731961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:03.731961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:03.731961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:03.731961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:03.731961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:03.735207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:03.735207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:03.735207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:21:03.735207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:21:03.735207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:21:03.735207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:21:03.735207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:21:03.735207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:21:03.735207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:21:03.735207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:03.735207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 396/811] 10.222.0.83 () {56 vars in 1093 bytes} [Mon Aug 1 15:21:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:03.735207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1510e-dc95-4f73-b822-c698fc1a6674 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:03.824174 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3260f-1af8-40fb-977a-ddca02527a28 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:03.850811 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1510e-dc95-4f73-b822-c698fc1a6674 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:03.862644 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea1c03-9944-4f03-b815-2836c17cbe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:03.863504 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea1c03-9944-4f03-b815-2836c17cbe35 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:03.863715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea1c03-9944-4f03-b815-2836c17cbe35 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:03.863917 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea1c03-9944-4f03-b815-2836c17cbe35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:03.876904 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea1c03-9944-4f03-b815-2836c17cbe35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:03.878451 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea1c03-9944-4f03-b815-2836c17cbe35 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:03.890934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd5717-9a38-4e1c-8f8b-fdd612233ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:03.891572 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd5717-9a38-4e1c-8f8b-fdd612233ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:03.891856 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd5717-9a38-4e1c-8f8b-fdd612233ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:03.892217 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd5717-9a38-4e1c-8f8b-fdd612233ea2 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:03.892907 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd5717-9a38-4e1c-8f8b-fdd612233ea2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:03.893211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd5717-9a38-4e1c-8f8b-fdd612233ea2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:03.894809 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd5717-9a38-4e1c-8f8b-fdd612233ea2 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:03.901121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-37ea1c03-9944-4f03-b815-2836c17cbe35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:03.901121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:03.901121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:03.901121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:03.901121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:03.901121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:03.901121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:03.901121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:03.901121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:03.901121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:03.901121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:03.901121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:03.901121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:03.901121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:03.901121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:03.904186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:03.904186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:03.904186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:03.904186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:03.904186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:03.904186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:03.904186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:03.904186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:03.904186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:03.904186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:03.904186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:03.904186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:03.904186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:03.904186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:03.904186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:03.904186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:03.904186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:03.909827 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:03.909827 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:03.909827 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:03.909827 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:03.910593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 397/812] 10.222.0.83 () {56 vars in 1124 bytes} [Mon Aug 1 15:21:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:03.910593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd5717-9a38-4e1c-8f8b-fdd612233ea2 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:03.910972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 416/813] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:03.910972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea1c03-9944-4f03-b815-2836c17cbe35 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:03.921494 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:03.926472 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:04.031690 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea1c03-9944-4f03-b815-2836c17cbe35 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:04.034700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd5717-9a38-4e1c-8f8b-fdd612233ea2 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:04.077971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc9b15-d4a2-48f9-9507-f0714f2c89d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:04.078756 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fd818-909c-4ec8-9594-b0d2b03ab657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:04.079145 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc9b15-d4a2-48f9-9507-f0714f2c89d9 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:04.079604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc9b15-d4a2-48f9-9507-f0714f2c89d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:04.079708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fd818-909c-4ec8-9594-b0d2b03ab657 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:04.080116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fd818-909c-4ec8-9594-b0d2b03ab657 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:04.080352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc9b15-d4a2-48f9-9507-f0714f2c89d9 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:04.080553 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fd818-909c-4ec8-9594-b0d2b03ab657 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:04.081508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc9b15-d4a2-48f9-9507-f0714f2c89d9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:04.081772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fd818-909c-4ec8-9594-b0d2b03ab657 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:04.082320 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc9b15-d4a2-48f9-9507-f0714f2c89d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:04.082444 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fd818-909c-4ec8-9594-b0d2b03ab657 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:04.084165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc9b15-d4a2-48f9-9507-f0714f2c89d9 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:04.084746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fd818-909c-4ec8-9594-b0d2b03ab657 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:04.093856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 417/814] 10.222.0.83 () {56 vars in 1124 bytes} [Mon Aug 1 15:21:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:04.096083 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fd818-909c-4ec8-9594-b0d2b03ab657 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:04.104842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 398/815] 10.222.0.83 () {56 vars in 1133 bytes} [Mon Aug 1 15:21:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:04.106034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc9b15-d4a2-48f9-9507-f0714f2c89d9 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:04.183849 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:04.209702 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:04.222295 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc9b15-d4a2-48f9-9507-f0714f2c89d9 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:04.254479 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fd818-909c-4ec8-9594-b0d2b03ab657 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:04.272371 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26ce75-8e9d-49a1-aa7b-d4a1f05ace1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:04.273800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26ce75-8e9d-49a1-aa7b-d4a1f05ace1d None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:04.274306 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26ce75-8e9d-49a1-aa7b-d4a1f05ace1d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:04.274738 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26ce75-8e9d-49a1-aa7b-d4a1f05ace1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:04.275841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26ce75-8e9d-49a1-aa7b-d4a1f05ace1d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:04.278979 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26ce75-8e9d-49a1-aa7b-d4a1f05ace1d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:04.292788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1406b55-026e-42f9-ad56-1c445f055648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:04.293337 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1406b55-026e-42f9-ad56-1c445f055648 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:04.293624 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1406b55-026e-42f9-ad56-1c445f055648 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:04.293941 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1406b55-026e-42f9-ad56-1c445f055648 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:04.294759 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1406b55-026e-42f9-ad56-1c445f055648 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:04.295107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1406b55-026e-42f9-ad56-1c445f055648 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:04.296519 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1406b55-026e-42f9-ad56-1c445f055648 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:04.321924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 418/816] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:04.321924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1406b55-026e-42f9-ad56-1c445f055648 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:04.322226 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 399/817] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:21:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:21:04.322226 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d52df-4abc-407c-969e-9cfaeebe750a None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:04.322226 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d52df-4abc-407c-969e-9cfaeebe750a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:04.322226 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d52df-4abc-407c-969e-9cfaeebe750a None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:04.322226 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 400/818] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:04.328200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf45aeb-2233-4037-a5c5-9280dcf38f05 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:04.328709 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf45aeb-2233-4037-a5c5-9280dcf38f05 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:04.329133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf45aeb-2233-4037-a5c5-9280dcf38f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:04.359877 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-0cf45aeb-2233-4037-a5c5-9280dcf38f05 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:04.363511 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0cf45aeb-2233-4037-a5c5-9280dcf38f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:04.483967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1406b55-026e-42f9-ad56-1c445f055648 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:04.520699 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 401/819] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:21:04.522350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:04.522707 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf45aeb-2233-4037-a5c5-9280dcf38f05 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:04.532159 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6126b0-2501-43d8-8570-18af8e7a0088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:04.533208 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6126b0-2501-43d8-8570-18af8e7a0088 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:04.533710 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6126b0-2501-43d8-8570-18af8e7a0088 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:04.534293 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6126b0-2501-43d8-8570-18af8e7a0088 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:04.599814 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:04.636362 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6126b0-2501-43d8-8570-18af8e7a0088 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:04.638391 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6126b0-2501-43d8-8570-18af8e7a0088 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:04.657382 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf45aeb-2233-4037-a5c5-9280dcf38f05 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:04.745834 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfc7f7-d7e0-4aaa-89e1-aa137163712e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:04.746561 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfc7f7-d7e0-4aaa-89e1-aa137163712e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:04.746881 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfc7f7-d7e0-4aaa-89e1-aa137163712e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:04.750488 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfc7f7-d7e0-4aaa-89e1-aa137163712e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:04.768201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfc7f7-d7e0-4aaa-89e1-aa137163712e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:04.773444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfc7f7-d7e0-4aaa-89e1-aa137163712e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:04.786063 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 419/820] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:21:04 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:21:04.787121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6126b0-2501-43d8-8570-18af8e7a0088 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:04.804803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-74dfc7f7-d7e0-4aaa-89e1-aa137163712e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:04.804803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:04.804803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:04.804803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:04.804803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:04.804803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:04.804803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:04.804803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:04.804803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:04.804803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:04.804803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:04.804803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:04.804803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:21:04.804803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:21:04.804803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:21:04.806731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:21:04.806731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:04.806731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:04.806731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:04.806731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:04.806731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:04.806731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:04.806731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:04.806731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:04.806731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:04.806731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:04.806731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:04.806731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:04.806731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:04.806731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:04.806731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:04.806731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:04.808471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:21:04.808471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:21:04.808471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:21:04.808471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:21:04.808471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:21:04.808471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:21:04.808471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:04.808471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:04.808471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 402/821] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:21:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:04.810799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfc7f7-d7e0-4aaa-89e1-aa137163712e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:04.837157 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:04.919666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6126b0-2501-43d8-8570-18af8e7a0088 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:04.942838 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfc7f7-d7e0-4aaa-89e1-aa137163712e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:04.957606 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:04.960815 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558a680-7f20-46d6-9b56-08b5fae6418c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:04.962149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558a680-7f20-46d6-9b56-08b5fae6418c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:04.962442 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558a680-7f20-46d6-9b56-08b5fae6418c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:04.963186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558a680-7f20-46d6-9b56-08b5fae6418c None admin] PATH_INFO: `/v3/services/network` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:04.963925 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558a680-7f20-46d6-9b56-08b5fae6418c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:04.965383 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558a680-7f20-46d6-9b56-08b5fae6418c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:04.975916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-7558a680-7f20-46d6-9b56-08b5fae6418c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:21:04.975916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:04.975916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:21:04.975916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:21:04.975916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:21:04.975916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:21:04.975916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:21:04.975916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:04.975916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:21:04.975916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:04.975916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:04.975916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:04.975916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:04.975916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:04.975916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:04.975916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:04.975916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:04.978281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:04.978281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:04.978281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:04.978281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:04.978281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:04.978281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:04.978281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:21:04.978281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:21:04.978281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:21:04.978281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:21:04.978281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:04.978281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:04.978281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:04.978281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:04.978281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:04.978281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:04.985833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:04.985833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:04.985833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:04.985833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:04.985833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:04.985833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:04.985833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:04.985833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:04.985833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:04.985833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:04.985833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:21:04.985833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:04.985833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:21:04.985833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:04.985833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 420/822] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:21:04 2022] GET /identity/v3/services/network => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:04.988974 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558a680-7f20-46d6-9b56-08b5fae6418c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:04.989120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd184ba-f53c-4130-8d65-0c413528441a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:04.989120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd184ba-f53c-4130-8d65-0c413528441a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:04.989120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd184ba-f53c-4130-8d65-0c413528441a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:04.989120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd184ba-f53c-4130-8d65-0c413528441a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:05.008752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd184ba-f53c-4130-8d65-0c413528441a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:05.011092 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd184ba-f53c-4130-8d65-0c413528441a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:05.032887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-3bd184ba-f53c-4130-8d65-0c413528441a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:05.032887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:05.032887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:05.032887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:05.032887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:05.032887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:05.032887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:05.032887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:05.032887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:05.032887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:05.032887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:05.032887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:05.032887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:05.032887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:05.032887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:05.034804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:05.034804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:05.034804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:05.034804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:05.034804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:05.034804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:05.034804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:05.034804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:05.034804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:05.034804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:05.034804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:05.034804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:05.034804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:05.034804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:05.034804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:05.034804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:05.034804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:05.036543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:05.036543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:05.036543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:05.036543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:05.036543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 403/823] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:05.036543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd184ba-f53c-4130-8d65-0c413528441a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:05.115078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:05.123615 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558a680-7f20-46d6-9b56-08b5fae6418c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:05.169045 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fad71-97fe-4738-99ed-fa306317f7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:05.170335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fad71-97fe-4738-99ed-fa306317f7fd None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:05.170822 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fad71-97fe-4738-99ed-fa306317f7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:05.171365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fad71-97fe-4738-99ed-fa306317f7fd None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:05.172637 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fad71-97fe-4738-99ed-fa306317f7fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:05.173172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fad71-97fe-4738-99ed-fa306317f7fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:05.175416 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fad71-97fe-4738-99ed-fa306317f7fd None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:05.190628 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 421/824] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:21:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:05.191931 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fad71-97fe-4738-99ed-fa306317f7fd None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:05.232587 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd184ba-f53c-4130-8d65-0c413528441a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:05.299606 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ff064-aca6-4932-bab7-f884d2a70c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:05.300514 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ff064-aca6-4932-bab7-f884d2a70c4d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:05.301013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ff064-aca6-4932-bab7-f884d2a70c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:05.301503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ff064-aca6-4932-bab7-f884d2a70c4d None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:05.303067 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ff064-aca6-4932-bab7-f884d2a70c4d None admin] RBAC: Adding query filter params (name=network) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:05.303559 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ff064-aca6-4932-bab7-f884d2a70c4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:05.306352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ff064-aca6-4932-bab7-f884d2a70c4d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:05.310585 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fad71-97fe-4738-99ed-fa306317f7fd None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:05.322420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 404/825] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Aug 1 15:21:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:05.324196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ff064-aca6-4932-bab7-f884d2a70c4d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:05.352551 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdd7c3-d679-4a8d-9b13-6aa9b4727db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:05.353540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdd7c3-d679-4a8d-9b13-6aa9b4727db0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:05.354063 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdd7c3-d679-4a8d-9b13-6aa9b4727db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:05.354599 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdd7c3-d679-4a8d-9b13-6aa9b4727db0 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:05.355556 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdd7c3-d679-4a8d-9b13-6aa9b4727db0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:05.356169 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdd7c3-d679-4a8d-9b13-6aa9b4727db0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:05.357845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdd7c3-d679-4a8d-9b13-6aa9b4727db0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:05.371722 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 422/826] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:05.373092 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdd7c3-d679-4a8d-9b13-6aa9b4727db0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:05.379445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:05.395256 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:05.494142 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdd7c3-d679-4a8d-9b13-6aa9b4727db0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:05.521597 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ff064-aca6-4932-bab7-f884d2a70c4d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:05.538047 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71d5d5-39f4-4c41-a412-29ac2c30b4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:05.539226 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71d5d5-39f4-4c41-a412-29ac2c30b4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:05.539877 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71d5d5-39f4-4c41-a412-29ac2c30b4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:05.540471 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71d5d5-39f4-4c41-a412-29ac2c30b4b8 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:05.541528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb71d5d5-39f4-4c41-a412-29ac2c30b4b8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:05.542138 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb71d5d5-39f4-4c41-a412-29ac2c30b4b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:05.544253 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb71d5d5-39f4-4c41-a412-29ac2c30b4b8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:05.550962 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 423/827] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Aug 1 15:21:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:05.552209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71d5d5-39f4-4c41-a412-29ac2c30b4b8 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:05.585949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27255aea-2f59-4c83-ad8b-a5900172935f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:05.586675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27255aea-2f59-4c83-ad8b-a5900172935f None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:05.586979 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27255aea-2f59-4c83-ad8b-a5900172935f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:05.587256 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27255aea-2f59-4c83-ad8b-a5900172935f None admin] PATH_INFO: `/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/6873a3e440bd4b63b0082fc85beef49c/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:05.589561 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27255aea-2f59-4c83-ad8b-a5900172935f None admin] RBAC: Authorizing `identity:create_grant(project_id=1f4e27cc50bd4f9f8ced4cc544ab3064, user_id=6873a3e440bd4b63b0082fc85beef49c, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:05.591181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27255aea-2f59-4c83-ad8b-a5900172935f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:05.608241 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 405/828] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:21:05 2022] PUT /identity/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/6873a3e440bd4b63b0082fc85beef49c/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:21:05.609135 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27255aea-2f59-4c83-ad8b-a5900172935f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:05.658494 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:05.681570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71d5d5-39f4-4c41-a412-29ac2c30b4b8 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:05.724436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed30f9-3d43-44d0-a4c7-10b1ba4a56c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:05.726009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed30f9-3d43-44d0-a4c7-10b1ba4a56c2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:05.726684 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed30f9-3d43-44d0-a4c7-10b1ba4a56c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:05.727562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed30f9-3d43-44d0-a4c7-10b1ba4a56c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:05.739102 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed30f9-3d43-44d0-a4c7-10b1ba4a56c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:05.741069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed30f9-3d43-44d0-a4c7-10b1ba4a56c2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:05.755929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-17ed30f9-3d43-44d0-a4c7-10b1ba4a56c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:05.755929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:05.755929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:05.755929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:05.755929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:05.755929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:05.755929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:05.755929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:05.755929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:05.755929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:05.755929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:05.755929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:05.755929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:21:05.755929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:21:05.755929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:21:05.756976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:21:05.756976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:05.756976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:05.756976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:05.756976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:05.756976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:05.756976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:05.756976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:05.756976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:05.756976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:05.756976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:05.756976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:05.756976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:05.756976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:05.756976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:05.756976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:05.756976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:05.758049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:21:05.758049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:21:05.758049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:21:05.758049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:21:05.758049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:21:05.758049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:21:05.758049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:05.758049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:05.761659 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 424/829] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:21:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:05.772241 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed30f9-3d43-44d0-a4c7-10b1ba4a56c2 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:05.815903 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:05.850951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27255aea-2f59-4c83-ad8b-a5900172935f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:05.948825 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfea6cb-73db-44c1-a4b8-3e9d07c9392f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:05.949480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfea6cb-73db-44c1-a4b8-3e9d07c9392f None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:05.949754 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfea6cb-73db-44c1-a4b8-3e9d07c9392f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:05.950068 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfea6cb-73db-44c1-a4b8-3e9d07c9392f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:05.950832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfea6cb-73db-44c1-a4b8-3e9d07c9392f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:05.952236 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfea6cb-73db-44c1-a4b8-3e9d07c9392f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:05.983857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 406/830] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:21:05 2022] POST /identity/v3/endpoints => generated 327 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:21:05.985501 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed30f9-3d43-44d0-a4c7-10b1ba4a56c2 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:06.032673 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c69adf-8f18-4bee-97b1-6191efd74a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:06.037153 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c69adf-8f18-4bee-97b1-6191efd74a4a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:06.037153 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c69adf-8f18-4bee-97b1-6191efd74a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:06.037153 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c69adf-8f18-4bee-97b1-6191efd74a4a None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:06.037153 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c69adf-8f18-4bee-97b1-6191efd74a4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:06.037153 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c69adf-8f18-4bee-97b1-6191efd74a4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:06.037153 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c69adf-8f18-4bee-97b1-6191efd74a4a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:06.039018 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:06.054032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 425/831] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:21:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:06.076586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfea6cb-73db-44c1-a4b8-3e9d07c9392f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:06.178948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af1d55-e494-4f02-92c1-065f1963a473 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:06.179400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af1d55-e494-4f02-92c1-065f1963a473 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:06.179844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af1d55-e494-4f02-92c1-065f1963a473 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:06.191370 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 426/832] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:06 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:21:06.204394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1608a-c887-4435-8583-903755cd0302 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:06.204845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1608a-c887-4435-8583-903755cd0302 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:06.205303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1608a-c887-4435-8583-903755cd0302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:06.206340 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfea6cb-73db-44c1-a4b8-3e9d07c9392f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:06.261372 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-4af1608a-c887-4435-8583-903755cd0302 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:06.262830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4af1608a-c887-4435-8583-903755cd0302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:06.296013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aacfa3-12c7-4687-9545-f6a986bd6db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:06.297043 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aacfa3-12c7-4687-9545-f6a986bd6db8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:06.297512 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aacfa3-12c7-4687-9545-f6a986bd6db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:06.298113 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aacfa3-12c7-4687-9545-f6a986bd6db8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:06.299634 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aacfa3-12c7-4687-9545-f6a986bd6db8 None admin] RBAC: Adding query filter params (role.id=7bdfa43dc1d046f7965ffe52b74bc58f, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=25dc41ffc41c47fdbeec06110d04610c) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:06.300293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aacfa3-12c7-4687-9545-f6a986bd6db8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:06.303151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aacfa3-12c7-4687-9545-f6a986bd6db8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:06.317322 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 407/833] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:21:06 2022] GET /identity/v3/role_assignments?user.id=25dc41ffc41c47fdbeec06110d04610c&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=7bdfa43dc1d046f7965ffe52b74bc58f => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:06.346503 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 427/834] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:06.387323 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aacfa3-12c7-4687-9545-f6a986bd6db8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:06.409202 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:06.600945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aacfa3-12c7-4687-9545-f6a986bd6db8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:06.667364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdfba3-79d9-47f0-987a-f8750c19ff16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:06.668474 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdfba3-79d9-47f0-987a-f8750c19ff16 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:06.669077 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdfba3-79d9-47f0-987a-f8750c19ff16 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:06.669557 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdfba3-79d9-47f0-987a-f8750c19ff16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:06.689206 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fdfba3-79d9-47f0-987a-f8750c19ff16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:21:06.700443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-15fdfba3-79d9-47f0-987a-f8750c19ff16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:21:06.700443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:06.700443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:06.700443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:06.700443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:06.700443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:06.700443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:06.700443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:06.700443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:06.700443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:06.700443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:06.700443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:06.700443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:21:06.700443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:21:06.700443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:21:06.700443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:21:06.703702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:21:06.703702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:21:06.703702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:21:06.703702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:21:06.703702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:06.703702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:06.703702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:06.703702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:06.703702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:06.703702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:06.703702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:06.703702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:06.703702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:06.703702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:06.703702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:06.703702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:06.705960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:06.705960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:06.705960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:06.705960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:06.705960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:21:06.705960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:21:06.705960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:21:06.705960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:21:06.705960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:21:06.705960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:21:06.705960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:21:06.705960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:06.705960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 408/835] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:21:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:06.721322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1608a-c887-4435-8583-903755cd0302 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:06.899747 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1608a-c887-4435-8583-903755cd0302 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:06.947378 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:06.966294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599d23a-6027-4d54-8d4b-c1bf8c7782ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:06.967410 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599d23a-6027-4d54-8d4b-c1bf8c7782ff None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:06.967904 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599d23a-6027-4d54-8d4b-c1bf8c7782ff None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:06.968376 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599d23a-6027-4d54-8d4b-c1bf8c7782ff None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:06.969681 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e599d23a-6027-4d54-8d4b-c1bf8c7782ff None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:06.970249 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e599d23a-6027-4d54-8d4b-c1bf8c7782ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:06.972779 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e599d23a-6027-4d54-8d4b-c1bf8c7782ff None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:06.986903 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 428/836] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:21:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:06.995532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdfba3-79d9-47f0-987a-f8750c19ff16 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:07.126826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:07.188067 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdfba3-79d9-47f0-987a-f8750c19ff16 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:07.249173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c008597-0862-4062-8941-9d7a4f722942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:07.250291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c008597-0862-4062-8941-9d7a4f722942 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:07.250853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c008597-0862-4062-8941-9d7a4f722942 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:07.251342 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c008597-0862-4062-8941-9d7a4f722942 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:07.264535 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e8084-5259-45b5-a9a5-54213c8b3aed None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:07.264980 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e8084-5259-45b5-a9a5-54213c8b3aed None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:07.265440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e8084-5259-45b5-a9a5-54213c8b3aed None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:07.268153 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 429/837] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:21:07.271573 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c008597-0862-4062-8941-9d7a4f722942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:07.275464 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c008597-0862-4062-8941-9d7a4f722942 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:07.276841 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e3a41-0059-4502-92dc-1319ebf39770 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:07.277307 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e3a41-0059-4502-92dc-1319ebf39770 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:07.277763 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e3a41-0059-4502-92dc-1319ebf39770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:07.302001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-5c008597-0862-4062-8941-9d7a4f722942 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:07.302001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:07.302001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:07.302001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:07.302001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:07.302001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:07.302001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:07.302001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:07.302001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:07.302001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:07.302001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:07.302001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:07.302001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:07.302001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:07.302001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:07.303528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:07.303528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:07.303528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:07.303528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:07.303528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:07.303528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:07.303528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:07.303528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:07.303528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:07.303528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:07.303528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:07.303528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:07.303528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:07.303528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:07.303528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:07.303528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:07.303528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:07.305315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:07.305315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:07.305315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:07.305315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:07.305750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-963e3a41-0059-4502-92dc-1319ebf39770 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:07.307000 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-963e3a41-0059-4502-92dc-1319ebf39770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:07.309336 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 409/838] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:07.318109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c008597-0862-4062-8941-9d7a4f722942 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:07.410078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 430/839] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:21:07.440010 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e3a41-0059-4502-92dc-1319ebf39770 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:07.516636 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c008597-0862-4062-8941-9d7a4f722942 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:07.547281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:07.569988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22c49c-ab34-4535-9d8d-3732e89e1ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:07.572019 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22c49c-ab34-4535-9d8d-3732e89e1ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:07.572875 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22c49c-ab34-4535-9d8d-3732e89e1ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:07.573408 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22c49c-ab34-4535-9d8d-3732e89e1ca9 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:07.574376 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22c49c-ab34-4535-9d8d-3732e89e1ca9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:07.574902 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22c49c-ab34-4535-9d8d-3732e89e1ca9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:07.576602 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22c49c-ab34-4535-9d8d-3732e89e1ca9 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:07.595359 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 410/840] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:07.602171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e3a41-0059-4502-92dc-1319ebf39770 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:07.602423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22c49c-ab34-4535-9d8d-3732e89e1ca9 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:07.635516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:07.674553 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822d8f2-bb03-4f00-b4a4-2c6170da3fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:07.675998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822d8f2-bb03-4f00-b4a4-2c6170da3fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:07.677472 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822d8f2-bb03-4f00-b4a4-2c6170da3fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:07.678148 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822d8f2-bb03-4f00-b4a4-2c6170da3fb3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:07.680762 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822d8f2-bb03-4f00-b4a4-2c6170da3fb3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:07.683584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822d8f2-bb03-4f00-b4a4-2c6170da3fb3 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:07.705514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-8822d8f2-bb03-4f00-b4a4-2c6170da3fb3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:21:07.705514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:07.705514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:21:07.705514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:21:07.705514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:21:07.705514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:21:07.705514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:21:07.705514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:07.705514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:21:07.705514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:07.705514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:07.705514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:07.705514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:07.705514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:07.705514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:07.705514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:07.705514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:07.707396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:07.707396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:07.707396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:07.707396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:07.707396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:07.707396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:07.707396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:21:07.707396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:21:07.707396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:21:07.707396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:21:07.707396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:07.707396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:07.707396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:07.707396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:07.707396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:07.707396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:07.708936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:07.708936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:07.708936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:07.708936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:07.708936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:07.708936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:07.708936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:07.708936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:07.708936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:07.708936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:07.708936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:21:07.708936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:07.708936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:21:07.708936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:07.712751 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 431/841] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:21:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:07.722139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822d8f2-bb03-4f00-b4a4-2c6170da3fb3 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:07.748106 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22c49c-ab34-4535-9d8d-3732e89e1ca9 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:07.807735 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830bc9d-b3ee-4903-9526-9076a8da6e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:07.809153 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830bc9d-b3ee-4903-9526-9076a8da6e13 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:07.809870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830bc9d-b3ee-4903-9526-9076a8da6e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:07.810612 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830bc9d-b3ee-4903-9526-9076a8da6e13 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:07.824898 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830bc9d-b3ee-4903-9526-9076a8da6e13 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:07.827706 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830bc9d-b3ee-4903-9526-9076a8da6e13 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:07.842041 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822d8f2-bb03-4f00-b4a4-2c6170da3fb3 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:07.848865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-e830bc9d-b3ee-4903-9526-9076a8da6e13 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:21:07.848865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:07.848865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:07.848865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:07.848865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:07.848865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:07.848865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:07.848865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:07.848865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:07.848865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:07.848865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:07.848865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:07.848865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:21:07.848865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:21:07.848865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:21:07.848865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:21:07.850441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:07.850441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:07.850441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:21:07.850441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:07.850441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:21:07.850441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:07.850441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:07.850441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:07.850441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:07.850441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:07.850441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:07.850441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:07.850441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:07.850441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:07.850441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:07.850441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:07.850441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:07.850441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:07.852184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:07.852184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:07.852184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:21:07.852184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:21:07.852184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:21:07.852184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:21:07.852184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:21:07.852184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:21:07.852184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:21:07.852184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:07.856188 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 411/842] 10.222.0.83 () {56 vars in 1117 bytes} [Mon Aug 1 15:21:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:07.864646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830bc9d-b3ee-4903-9526-9076a8da6e13 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:07.892692 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259cd84-504c-482c-bdf5-5a3687fac376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:07.893694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259cd84-504c-482c-bdf5-5a3687fac376 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:07.894247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259cd84-504c-482c-bdf5-5a3687fac376 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:07.894726 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259cd84-504c-482c-bdf5-5a3687fac376 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:07.896702 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259cd84-504c-482c-bdf5-5a3687fac376 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:07.897418 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259cd84-504c-482c-bdf5-5a3687fac376 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:07.899225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259cd84-504c-482c-bdf5-5a3687fac376 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:07.906746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 432/843] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:21:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:07.914566 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259cd84-504c-482c-bdf5-5a3687fac376 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:07.919840 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:07.968136 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:07.989931 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830bc9d-b3ee-4903-9526-9076a8da6e13 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:08.029563 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61442955-ec8f-4b0d-8ba4-11f73ef0dec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:08.031101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61442955-ec8f-4b0d-8ba4-11f73ef0dec4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:08.031533 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61442955-ec8f-4b0d-8ba4-11f73ef0dec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:08.031985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61442955-ec8f-4b0d-8ba4-11f73ef0dec4 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:08.032943 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61442955-ec8f-4b0d-8ba4-11f73ef0dec4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:08.033403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61442955-ec8f-4b0d-8ba4-11f73ef0dec4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:08.034750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259cd84-504c-482c-bdf5-5a3687fac376 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:08.035491 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61442955-ec8f-4b0d-8ba4-11f73ef0dec4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:08.057017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 412/844] 10.222.0.83 () {56 vars in 1149 bytes} [Mon Aug 1 15:21:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:08.065247 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61442955-ec8f-4b0d-8ba4-11f73ef0dec4 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:08.082006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33232141-95f8-4244-aeb0-56a9630e2af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:08.082648 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33232141-95f8-4244-aeb0-56a9630e2af6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:08.082975 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33232141-95f8-4244-aeb0-56a9630e2af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:08.083296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33232141-95f8-4244-aeb0-56a9630e2af6 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:08.083998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33232141-95f8-4244-aeb0-56a9630e2af6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:08.084307 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33232141-95f8-4244-aeb0-56a9630e2af6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:08.085797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33232141-95f8-4244-aeb0-56a9630e2af6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:08.098072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 433/845] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:21:07 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:08.204583 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61442955-ec8f-4b0d-8ba4-11f73ef0dec4 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:08.206624 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:08.247300 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916d62a-3699-4600-846d-81a4845de3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:08.247870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916d62a-3699-4600-846d-81a4845de3bf None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:08.248150 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916d62a-3699-4600-846d-81a4845de3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:08.248426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916d62a-3699-4600-846d-81a4845de3bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:08.326344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916d62a-3699-4600-846d-81a4845de3bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:08.332019 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916d62a-3699-4600-846d-81a4845de3bf None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:08.438371 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:08.443901 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 413/846] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:21:08 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:21:08.450583 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33232141-95f8-4244-aeb0-56a9630e2af6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:08.567237 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5241c-9a5e-4e27-bf37-03e5eab19f09 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:08.567545 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5241c-9a5e-4e27-bf37-03e5eab19f09 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:08.567731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5241c-9a5e-4e27-bf37-03e5eab19f09 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:08.569422 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 414/847] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:08.575141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33232141-95f8-4244-aeb0-56a9630e2af6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:08.575347 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c5b9f-b05e-41e2-bfe5-e68932b72e77 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:08.575570 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c5b9f-b05e-41e2-bfe5-e68932b72e77 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:08.575856 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c5b9f-b05e-41e2-bfe5-e68932b72e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:08.588673 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:08.595994 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-143c5b9f-b05e-41e2-bfe5-e68932b72e77 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:08.597405 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-143c5b9f-b05e-41e2-bfe5-e68932b72e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:08.615936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa9ab2-a89d-4e13-8931-467c89b90274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:08.616651 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa9ab2-a89d-4e13-8931-467c89b90274 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:08.616941 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa9ab2-a89d-4e13-8931-467c89b90274 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:08.617221 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa9ab2-a89d-4e13-8931-467c89b90274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:08.634183 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa9ab2-a89d-4e13-8931-467c89b90274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:08.635662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa9ab2-a89d-4e13-8931-467c89b90274 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:08.651675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-a4aa9ab2-a89d-4e13-8931-467c89b90274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:08.651675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:08.651675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:08.651675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:08.651675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:08.651675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:08.651675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:08.651675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:08.651675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:08.651675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:08.651675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:08.651675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:08.651675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:08.651675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:08.651675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:08.652650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:08.652650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:08.652650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:08.652650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:08.652650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:08.652650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:08.652650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:08.652650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:08.652650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:08.652650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:08.652650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:08.652650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:08.652650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:08.652650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:08.652650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:08.652650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:08.652650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:08.653614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:08.653614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:08.653614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:08.653614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:08.657717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 434/848] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:08.659143 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa9ab2-a89d-4e13-8931-467c89b90274 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:08.693779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 415/849] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:21:08.718958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c5b9f-b05e-41e2-bfe5-e68932b72e77 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:08.783387 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa9ab2-a89d-4e13-8931-467c89b90274 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:08.824680 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53d132-de88-406d-a0bb-21ce502d8f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:08.825231 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53d132-de88-406d-a0bb-21ce502d8f4a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:08.825511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53d132-de88-406d-a0bb-21ce502d8f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:08.825785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53d132-de88-406d-a0bb-21ce502d8f4a None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:08.826532 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f53d132-de88-406d-a0bb-21ce502d8f4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:08.826840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f53d132-de88-406d-a0bb-21ce502d8f4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:08.828216 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f53d132-de88-406d-a0bb-21ce502d8f4a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:08.834046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c5b9f-b05e-41e2-bfe5-e68932b72e77 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:08.843023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 435/850] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:08.844413 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096235f-7b2c-40bd-bc79-a410671e0ed2 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:08.844706 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096235f-7b2c-40bd-bc79-a410671e0ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:08.844980 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096235f-7b2c-40bd-bc79-a410671e0ed2 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:08.846582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 436/851] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:08.851146 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096235f-7b2c-40bd-bc79-a410671e0ed2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:08.872730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:08.877816 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:08.882707 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168821f-1192-40b2-aee5-9b37fe519e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:08.883343 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168821f-1192-40b2-aee5-9b37fe519e81 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:08.883675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168821f-1192-40b2-aee5-9b37fe519e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:08.883924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168821f-1192-40b2-aee5-9b37fe519e81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:08.897269 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1168821f-1192-40b2-aee5-9b37fe519e81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:21:08.903205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-1168821f-1192-40b2-aee5-9b37fe519e81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:21:08.903205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:08.903205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:08.903205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:08.903205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:08.903205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:08.903205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:08.903205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:08.903205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:08.903205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:08.903205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:08.903205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:08.903205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:21:08.903205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:21:08.903205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:21:08.903205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:21:08.906197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:21:08.906197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:21:08.906197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:21:08.906197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:21:08.906197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:08.906197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:08.906197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:08.906197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:08.906197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:08.906197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:08.906197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:08.906197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:08.906197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:08.906197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:08.906197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:08.906197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:08.907180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:08.907180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:08.907180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:08.907180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:08.907180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:21:08.907180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:21:08.907180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:21:08.907180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:21:08.907180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:21:08.907180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:21:08.907180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:21:08.907180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:08.907180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 416/852] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:21:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:21:08.907180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e35b93-132d-444c-8a42-d2db1424961b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:08.908031 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e35b93-132d-444c-8a42-d2db1424961b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:08.908031 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e35b93-132d-444c-8a42-d2db1424961b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:08.927362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-62e35b93-132d-444c-8a42-d2db1424961b None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:08.928644 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-62e35b93-132d-444c-8a42-d2db1424961b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:08.983778 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096235f-7b2c-40bd-bc79-a410671e0ed2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:09.010087 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 417/853] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:09.010087 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e35b93-132d-444c-8a42-d2db1424961b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:09.037163 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a854aff-5861-4ac5-a25c-89aa3cd4a8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:09.037818 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a854aff-5861-4ac5-a25c-89aa3cd4a8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:09.038304 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a854aff-5861-4ac5-a25c-89aa3cd4a8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:09.038594 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a854aff-5861-4ac5-a25c-89aa3cd4a8d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:09.047950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a854aff-5861-4ac5-a25c-89aa3cd4a8d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:09.049440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a854aff-5861-4ac5-a25c-89aa3cd4a8d8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:09.065670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-0a854aff-5861-4ac5-a25c-89aa3cd4a8d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:09.065670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:09.065670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:09.065670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:09.065670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:09.065670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:09.065670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:09.065670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:09.065670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:09.065670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:09.065670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:09.065670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:09.065670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:21:09.065670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:21:09.065670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:21:09.066842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:21:09.066842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:09.066842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:09.066842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:09.066842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:09.066842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:09.066842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:09.066842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:09.066842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:09.066842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:09.066842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:09.066842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:09.066842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:09.066842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:09.066842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:09.066842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:09.066842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:09.067750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:21:09.067750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:21:09.067750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:21:09.067750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:21:09.067750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:21:09.067750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:21:09.067750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:09.067750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:09.067750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 437/854] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:21:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:09.068256 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a854aff-5861-4ac5-a25c-89aa3cd4a8d8 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:09.134883 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e35b93-132d-444c-8a42-d2db1424961b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:09.155638 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:09.169862 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:09.178366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363ef11-6156-49ef-966f-ca6a57111dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:09.179110 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363ef11-6156-49ef-966f-ca6a57111dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:09.179464 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363ef11-6156-49ef-966f-ca6a57111dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:09.179818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363ef11-6156-49ef-966f-ca6a57111dd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:09.180806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363ef11-6156-49ef-966f-ca6a57111dd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:09.181196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363ef11-6156-49ef-966f-ca6a57111dd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:09.182886 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363ef11-6156-49ef-966f-ca6a57111dd4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:09.192642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 418/855] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:21:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:09.198278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363ef11-6156-49ef-966f-ca6a57111dd4 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:09.201792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a854aff-5861-4ac5-a25c-89aa3cd4a8d8 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:09.247530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ef917-73e0-4783-9558-d48865026616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:09.248257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ef917-73e0-4783-9558-d48865026616 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:09.248541 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ef917-73e0-4783-9558-d48865026616 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:09.248817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ef917-73e0-4783-9558-d48865026616 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:09.249611 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ef917-73e0-4783-9558-d48865026616 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:09.251192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ef917-73e0-4783-9558-d48865026616 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:09.263017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-ce1ef917-73e0-4783-9558-d48865026616 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:21:09.263017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:09.263017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:21:09.263017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:21:09.263017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:21:09.263017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:21:09.263017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:21:09.263017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:09.263017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:21:09.263017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:09.263017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:09.263017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:09.263017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:09.263017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:09.263017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:09.263017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:09.264574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:09.264574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:09.264574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:09.264574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:09.264574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:09.264574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:09.264574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:09.264574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:21:09.264574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:21:09.264574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:21:09.264574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:21:09.264574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:09.264574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:09.264574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:09.264574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:09.264574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:09.266682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:09.266682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:09.266682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:09.266682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:09.266682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:09.266682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:09.266682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:09.266682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:09.266682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:09.266682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:09.266682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:09.266682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:21:09.266682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:09.266682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:21:09.266682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:09.266682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 438/856] 10.222.0.83 () {56 vars in 1132 bytes} [Mon Aug 1 15:21:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:09.268056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ef917-73e0-4783-9558-d48865026616 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:09.326542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363ef11-6156-49ef-966f-ca6a57111dd4 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:09.381497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9b96f-0777-456c-bf4b-a4c31c7016b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:09.382471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9b96f-0777-456c-bf4b-a4c31c7016b2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:09.382904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9b96f-0777-456c-bf4b-a4c31c7016b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:09.383819 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9b96f-0777-456c-bf4b-a4c31c7016b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:09.385106 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9b96f-0777-456c-bf4b-a4c31c7016b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:09.385511 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9b96f-0777-456c-bf4b-a4c31c7016b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:09.387994 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9b96f-0777-456c-bf4b-a4c31c7016b2 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:09.392846 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ef917-73e0-4783-9558-d48865026616 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:09.413073 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 419/857] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:21:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:09.413073 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9b96f-0777-456c-bf4b-a4c31c7016b2 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:09.450011 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95a508-9797-4559-90a1-144875c9925b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:09.450968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95a508-9797-4559-90a1-144875c9925b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:09.451397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95a508-9797-4559-90a1-144875c9925b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:09.451810 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95a508-9797-4559-90a1-144875c9925b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:09.458128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:09.470536 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d95a508-9797-4559-90a1-144875c9925b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:09.472931 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d95a508-9797-4559-90a1-144875c9925b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:09.477668 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:09.494465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-5d95a508-9797-4559-90a1-144875c9925b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:09.494465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:09.494465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:09.494465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:09.494465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:09.494465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:09.494465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:09.494465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:09.494465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:09.494465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:09.494465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:09.494465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:09.494465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:09.494465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:09.494465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:09.495568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:09.495568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:09.495568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:09.495568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:09.495568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:09.495568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:09.495568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:09.495568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:09.495568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:09.495568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:09.495568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:09.495568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:09.495568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:09.495568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:09.495568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:09.495568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:09.495568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:09.496506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:09.496506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:09.496506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:09.496506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:09.496506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 439/858] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:09.496506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95a508-9797-4559-90a1-144875c9925b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:09.537340 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9b96f-0777-456c-bf4b-a4c31c7016b2 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:09.577030 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee61bc-459f-4519-a4dc-9e602e1df2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:09.577664 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee61bc-459f-4519-a4dc-9e602e1df2af None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:09.578001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee61bc-459f-4519-a4dc-9e602e1df2af None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:09.578284 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee61bc-459f-4519-a4dc-9e602e1df2af None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:09.579234 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee61bc-459f-4519-a4dc-9e602e1df2af None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:09.579563 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee61bc-459f-4519-a4dc-9e602e1df2af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:09.581387 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee61bc-459f-4519-a4dc-9e602e1df2af None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:09.587869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 420/859] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Aug 1 15:21:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:09.593234 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee61bc-459f-4519-a4dc-9e602e1df2af None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:09.619480 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95a508-9797-4559-90a1-144875c9925b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:09.662652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f10a24-95c2-469f-9b16-dc86ee05811a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:09.663463 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f10a24-95c2-469f-9b16-dc86ee05811a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:09.663943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f10a24-95c2-469f-9b16-dc86ee05811a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:09.664376 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f10a24-95c2-469f-9b16-dc86ee05811a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:09.665357 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f10a24-95c2-469f-9b16-dc86ee05811a None admin] RBAC: Adding query filter params (role.id=091d39715077464daa4cf890db55e6b9, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=f36c6987ef774a1684c55f885e2a4f67) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:09.665912 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f10a24-95c2-469f-9b16-dc86ee05811a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:09.667820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f10a24-95c2-469f-9b16-dc86ee05811a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:09.685927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 440/860] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:21:09 2022] GET /identity/v3/role_assignments?user.id=f36c6987ef774a1684c55f885e2a4f67&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=091d39715077464daa4cf890db55e6b9 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:09.685927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f10a24-95c2-469f-9b16-dc86ee05811a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:09.753638 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee61bc-459f-4519-a4dc-9e602e1df2af None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:09.758951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:09.819506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:09.824461 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8181d-c205-44c6-a198-cd557d745192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:09.825580 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8181d-c205-44c6-a198-cd557d745192 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:09.826175 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8181d-c205-44c6-a198-cd557d745192 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:09.826712 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8181d-c205-44c6-a198-cd557d745192 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:09.828031 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8181d-c205-44c6-a198-cd557d745192 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:09.828682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8181d-c205-44c6-a198-cd557d745192 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:09.832162 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8181d-c205-44c6-a198-cd557d745192 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:09.854247 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 421/861] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:09.863888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8181d-c205-44c6-a198-cd557d745192 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:09.911916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f10a24-95c2-469f-9b16-dc86ee05811a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:09.975546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9842bf0-460d-4ed4-b86e-6973f11ed6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:09.976089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9842bf0-460d-4ed4-b86e-6973f11ed6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:09.976383 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9842bf0-460d-4ed4-b86e-6973f11ed6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:09.976686 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9842bf0-460d-4ed4-b86e-6973f11ed6e4 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:09.978747 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9842bf0-460d-4ed4-b86e-6973f11ed6e4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:09.979805 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9842bf0-460d-4ed4-b86e-6973f11ed6e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:09.980970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9842bf0-460d-4ed4-b86e-6973f11ed6e4 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:09.990169 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 441/862] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Aug 1 15:21:09 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:10.033757 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8181d-c205-44c6-a198-cd557d745192 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:10.098055 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec8f49-3b4d-484b-9f85-cd6f71af2bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:10.099437 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec8f49-3b4d-484b-9f85-cd6f71af2bfe None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:10.100059 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec8f49-3b4d-484b-9f85-cd6f71af2bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:10.100634 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec8f49-3b4d-484b-9f85-cd6f71af2bfe None admin] PATH_INFO: `/v3/users/swift` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:10.115108 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec8f49-3b4d-484b-9f85-cd6f71af2bfe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:10.118792 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec8f49-3b4d-484b-9f85-cd6f71af2bfe None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:10.140916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-90ec8f49-3b4d-484b-9f85-cd6f71af2bfe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:21:10.140916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:10.140916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:10.140916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:10.140916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:10.140916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:10.140916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:10.140916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:10.140916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:10.140916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:10.140916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:10.140916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:10.140916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:21:10.140916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:21:10.140916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:21:10.140916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:21:10.143332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:10.143332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:10.143332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:21:10.143332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:10.143332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:21:10.143332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:10.143332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:10.143332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:10.143332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:10.143332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:10.143332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:10.143332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:10.143332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:10.143332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:10.143332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:10.143332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:10.143332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:10.143332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:10.145117 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:10.145117 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:10.145117 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:21:10.145117 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:21:10.145117 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:21:10.145117 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:21:10.145117 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:21:10.145117 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:21:10.145117 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:21:10.145117 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:10.148619 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 422/863] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:21:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:10.154598 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9842bf0-460d-4ed4-b86e-6973f11ed6e4 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:10.284975 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9842bf0-460d-4ed4-b86e-6973f11ed6e4 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:10.285226 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca338f71-5de1-4eee-ae9e-f242db4d17b7 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:10.285226 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca338f71-5de1-4eee-ae9e-f242db4d17b7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:10.285475 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca338f71-5de1-4eee-ae9e-f242db4d17b7 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:10.287159 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 423/864] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:21:10.293351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645bd8d-d2aa-457e-81b8-9c8677d87eee None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:10.293638 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645bd8d-d2aa-457e-81b8-9c8677d87eee None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:10.293949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645bd8d-d2aa-457e-81b8-9c8677d87eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:10.316263 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-b645bd8d-d2aa-457e-81b8-9c8677d87eee None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:10.318326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b645bd8d-d2aa-457e-81b8-9c8677d87eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:10.322141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:10.335783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a90ae-995c-4394-86ee-27292e9149c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:10.337302 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a90ae-995c-4394-86ee-27292e9149c8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:10.337591 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a90ae-995c-4394-86ee-27292e9149c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:10.337863 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a90ae-995c-4394-86ee-27292e9149c8 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:10.338804 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a90ae-995c-4394-86ee-27292e9149c8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:10.339112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a90ae-995c-4394-86ee-27292e9149c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:10.340680 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a90ae-995c-4394-86ee-27292e9149c8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:10.345802 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:10.364544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 442/865] 10.222.0.83 () {56 vars in 1135 bytes} [Mon Aug 1 15:21:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:10.373942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a90ae-995c-4394-86ee-27292e9149c8 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:10.420462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 424/866] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:21:10.443474 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645bd8d-d2aa-457e-81b8-9c8677d87eee None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:10.507430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a90ae-995c-4394-86ee-27292e9149c8 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:10.552163 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d57d43-f015-4d6e-a2de-4e66fb1d35c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:10.552708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d57d43-f015-4d6e-a2de-4e66fb1d35c3 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:10.553056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d57d43-f015-4d6e-a2de-4e66fb1d35c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:10.553365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d57d43-f015-4d6e-a2de-4e66fb1d35c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:10.578321 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645bd8d-d2aa-457e-81b8-9c8677d87eee None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:10.623196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:10.625140 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9dadfe-37bc-4a8a-8373-9d3a9cbb0b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:10.625791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9dadfe-37bc-4a8a-8373-9d3a9cbb0b42 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:10.626468 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9dadfe-37bc-4a8a-8373-9d3a9cbb0b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:10.627035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9dadfe-37bc-4a8a-8373-9d3a9cbb0b42 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:10.628065 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9dadfe-37bc-4a8a-8373-9d3a9cbb0b42 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:10.630128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9dadfe-37bc-4a8a-8373-9d3a9cbb0b42 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:10.642231 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d57d43-f015-4d6e-a2de-4e66fb1d35c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:10.645382 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d57d43-f015-4d6e-a2de-4e66fb1d35c3 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:10.650747 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 425/867] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Aug 1 15:21:10 2022] POST /identity/v3/services => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:21:10.786264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 443/868] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:21:10 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:21:10.791976 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9dadfe-37bc-4a8a-8373-9d3a9cbb0b42 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:10.814766 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:10.928546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9dadfe-37bc-4a8a-8373-9d3a9cbb0b42 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:10.967954 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d2b09-1be9-410e-9f49-083e8a545a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:10.968634 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d2b09-1be9-410e-9f49-083e8a545a21 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:10.969039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d2b09-1be9-410e-9f49-083e8a545a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:10.969447 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d2b09-1be9-410e-9f49-083e8a545a21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:10.980897 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d2b09-1be9-410e-9f49-083e8a545a21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:10.982415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d2b09-1be9-410e-9f49-083e8a545a21 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:11.001315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-124d2b09-1be9-410e-9f49-083e8a545a21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:11.001315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:11.001315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:11.001315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:11.001315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:11.001315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:11.001315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:11.001315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:11.001315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:11.001315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:11.001315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:11.001315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:11.001315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:11.001315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:11.001315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:11.003111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:11.003111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:11.003111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:11.003111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:11.003111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:11.003111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:11.003111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:11.003111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:11.003111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:11.003111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:11.003111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:11.003111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:11.003111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:11.003111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:11.003111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:11.003111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:11.003111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:11.004593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:11.004593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:11.004593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:11.004593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:11.004593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 426/869] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:11.010586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d57d43-f015-4d6e-a2de-4e66fb1d35c3 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:11.118628 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:11.142423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d57d43-f015-4d6e-a2de-4e66fb1d35c3 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:11.181877 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c25f3-24d0-45f5-88d7-b95d73204b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:11.182573 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c25f3-24d0-45f5-88d7-b95d73204b1c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:11.182833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c25f3-24d0-45f5-88d7-b95d73204b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:11.183063 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c25f3-24d0-45f5-88d7-b95d73204b1c None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:11.183757 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c25f3-24d0-45f5-88d7-b95d73204b1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:11.184061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c25f3-24d0-45f5-88d7-b95d73204b1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:11.185424 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c25f3-24d0-45f5-88d7-b95d73204b1c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:11.199969 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 444/870] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:11.205306 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d2b09-1be9-410e-9f49-083e8a545a21 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:11.282652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:11.328690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d2b09-1be9-410e-9f49-083e8a545a21 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:11.364943 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b7a4b-f49d-4ce6-9d23-2a97da2c7ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:11.365667 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b7a4b-f49d-4ce6-9d23-2a97da2c7ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:11.365979 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b7a4b-f49d-4ce6-9d23-2a97da2c7ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:11.366256 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b7a4b-f49d-4ce6-9d23-2a97da2c7ac6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:11.374248 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b7a4b-f49d-4ce6-9d23-2a97da2c7ac6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:11.375795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b7a4b-f49d-4ce6-9d23-2a97da2c7ac6 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:11.389109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-9b5b7a4b-f49d-4ce6-9d23-2a97da2c7ac6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:11.389109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:11.389109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:11.389109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:11.389109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:11.389109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:11.389109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:11.389109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:11.389109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:11.389109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:11.389109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:11.389109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:11.389109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:21:11.389109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:21:11.389109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:21:11.390173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:21:11.390173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:11.390173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:11.390173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:11.390173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:11.390173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:11.390173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:11.390173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:11.390173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:11.390173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:11.390173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:11.390173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:11.390173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:11.390173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:11.390173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:11.390173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:11.390173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:11.391093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:21:11.391093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:21:11.391093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:21:11.391093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:21:11.391093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:21:11.391093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:21:11.391093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:11.391093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:11.391093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 427/871] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:21:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:11.399508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c25f3-24d0-45f5-88d7-b95d73204b1c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:11.520425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c25f3-24d0-45f5-88d7-b95d73204b1c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:11.560776 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95312e61-f072-4b38-bcc8-1f3e71c6f613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:11.561450 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95312e61-f072-4b38-bcc8-1f3e71c6f613 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:11.561736 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95312e61-f072-4b38-bcc8-1f3e71c6f613 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:11.562047 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95312e61-f072-4b38-bcc8-1f3e71c6f613 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:11.562810 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95312e61-f072-4b38-bcc8-1f3e71c6f613 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:11.563121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95312e61-f072-4b38-bcc8-1f3e71c6f613 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:11.564717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95312e61-f072-4b38-bcc8-1f3e71c6f613 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:11.578487 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 445/872] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:21:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:11.584569 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b7a4b-f49d-4ce6-9d23-2a97da2c7ac6 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:11.702130 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b7a4b-f49d-4ce6-9d23-2a97da2c7ac6 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:11.727487 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:11.742122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef74f91-00e7-4e71-b7a6-acbe2f357946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:11.743301 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef74f91-00e7-4e71-b7a6-acbe2f357946 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:11.743591 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef74f91-00e7-4e71-b7a6-acbe2f357946 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:11.743931 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef74f91-00e7-4e71-b7a6-acbe2f357946 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:11.744735 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef74f91-00e7-4e71-b7a6-acbe2f357946 None admin] RBAC: Adding query filter params (role.id=7bdfa43dc1d046f7965ffe52b74bc58f, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=6873a3e440bd4b63b0082fc85beef49c) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:11.745054 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef74f91-00e7-4e71-b7a6-acbe2f357946 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:11.746551 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef74f91-00e7-4e71-b7a6-acbe2f357946 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:11.753371 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 428/873] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:21:11 2022] GET /identity/v3/role_assignments?user.id=6873a3e440bd4b63b0082fc85beef49c&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=7bdfa43dc1d046f7965ffe52b74bc58f => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:12.216408 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74164e12-a8f3-4a7e-91a4-069ef3e57ac6 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:12.217089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74164e12-a8f3-4a7e-91a4-069ef3e57ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:12.217089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74164e12-a8f3-4a7e-91a4-069ef3e57ac6 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:12.218904 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 446/874] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:12.224540 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f227f-09a5-416a-b0be-6eb256fd8e1b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:12.225085 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f227f-09a5-416a-b0be-6eb256fd8e1b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:12.225085 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f227f-09a5-416a-b0be-6eb256fd8e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:12.244890 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-597f227f-09a5-416a-b0be-6eb256fd8e1b None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:12.246119 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-597f227f-09a5-416a-b0be-6eb256fd8e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:12.274650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c489e37-673e-42fd-a8da-e0bb607416a9 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:12.274650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c489e37-673e-42fd-a8da-e0bb607416a9 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:12.274650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c489e37-673e-42fd-a8da-e0bb607416a9 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:12.275485 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 447/875] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:21:12.281350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bdb56-122d-43c0-a3f1-84d3f380037f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:12.281987 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bdb56-122d-43c0-a3f1-84d3f380037f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:12.282475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bdb56-122d-43c0-a3f1-84d3f380037f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:12.289035 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:12.301710 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-6d6bdb56-122d-43c0-a3f1-84d3f380037f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:12.303343 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6d6bdb56-122d-43c0-a3f1-84d3f380037f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:12.325938 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 429/876] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:12 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:12.346155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f227f-09a5-416a-b0be-6eb256fd8e1b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:12.388350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 448/877] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:12 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:21:12.411400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6bdb56-122d-43c0-a3f1-84d3f380037f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:12.459135 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f227f-09a5-416a-b0be-6eb256fd8e1b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:12.479240 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:12.501247 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78278ba1-3af6-4bc1-828a-6fb3ea998566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:12.502353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78278ba1-3af6-4bc1-828a-6fb3ea998566 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:12.502875 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78278ba1-3af6-4bc1-828a-6fb3ea998566 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:12.503454 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78278ba1-3af6-4bc1-828a-6fb3ea998566 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:12.516613 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78278ba1-3af6-4bc1-828a-6fb3ea998566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:21:12.522448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-78278ba1-3af6-4bc1-828a-6fb3ea998566 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:21:12.522448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:12.522448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:12.522448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:12.522448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:12.522448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:12.522448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:12.522448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:12.522448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:12.522448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:12.522448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:12.522448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:12.522448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:21:12.522448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:21:12.522448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:21:12.522448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:21:12.523800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:21:12.523800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:21:12.523800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:21:12.523800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:21:12.523800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:12.523800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:12.523800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:12.523800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:12.523800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:12.523800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:12.523800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:12.523800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:12.523800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:12.523800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:12.523800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:12.523800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:12.525338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:12.525338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:12.525338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:12.525338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:12.525338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:21:12.525338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:21:12.525338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:21:12.525338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:21:12.525338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:21:12.525338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:21:12.525338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:21:12.525338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:12.533053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 430/878] 10.222.0.83 () {56 vars in 1120 bytes} [Mon Aug 1 15:21:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:21:12.534332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78278ba1-3af6-4bc1-828a-6fb3ea998566 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:12.540898 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6bdb56-122d-43c0-a3f1-84d3f380037f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:12.585231 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:12.593804 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae8c9a-987d-438e-a764-553e87f839eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:12.594879 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae8c9a-987d-438e-a764-553e87f839eb None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:12.595173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae8c9a-987d-438e-a764-553e87f839eb None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:12.595441 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae8c9a-987d-438e-a764-553e87f839eb None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:12.596773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae8c9a-987d-438e-a764-553e87f839eb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:12.598607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae8c9a-987d-438e-a764-553e87f839eb None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:12.625950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 449/879] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Aug 1 15:21:12 2022] POST /identity/v3/services => generated 269 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:21:12.673191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78278ba1-3af6-4bc1-828a-6fb3ea998566 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:12.728738 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0f101-3e43-40b0-9cc0-da3f5214620d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:12.729469 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0f101-3e43-40b0-9cc0-da3f5214620d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:12.729757 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0f101-3e43-40b0-9cc0-da3f5214620d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:12.730123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0f101-3e43-40b0-9cc0-da3f5214620d None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:12.730962 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0f101-3e43-40b0-9cc0-da3f5214620d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:12.731284 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0f101-3e43-40b0-9cc0-da3f5214620d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:12.732764 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0f101-3e43-40b0-9cc0-da3f5214620d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:12.744045 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 431/880] 10.222.0.83 () {56 vars in 1115 bytes} [Mon Aug 1 15:21:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:12.751247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae8c9a-987d-438e-a764-553e87f839eb None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:12.881596 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae8c9a-987d-438e-a764-553e87f839eb None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:12.935372 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca25508-04d8-4068-a3d8-8c2f824d92d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:12.936317 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca25508-04d8-4068-a3d8-8c2f824d92d6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:12.936735 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca25508-04d8-4068-a3d8-8c2f824d92d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:12.937194 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca25508-04d8-4068-a3d8-8c2f824d92d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:12.955353 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca25508-04d8-4068-a3d8-8c2f824d92d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:12.957723 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca25508-04d8-4068-a3d8-8c2f824d92d6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:12.981990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-2ca25508-04d8-4068-a3d8-8c2f824d92d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:12.981990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:12.981990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:12.981990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:12.981990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:12.981990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:12.981990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:12.981990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:12.981990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:12.981990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:12.981990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:12.981990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:12.981990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:12.981990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:12.981990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:12.982986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:12.982986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:12.982986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:12.982986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:12.982986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:12.982986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:12.982986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:12.982986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:12.982986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:12.982986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:12.982986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:12.982986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:12.982986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:12.982986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:12.982986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:12.982986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:12.982986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:12.983997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:12.983997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:12.983997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:12.983997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:12.984236 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 450/881] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:12.991112 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0f101-3e43-40b0-9cc0-da3f5214620d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:13.038050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:13.109955 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0f101-3e43-40b0-9cc0-da3f5214620d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:13.147726 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff65a2-2563-4339-bb39-79371e02aa30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:13.148345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff65a2-2563-4339-bb39-79371e02aa30 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:13.149337 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff65a2-2563-4339-bb39-79371e02aa30 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:13.149622 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff65a2-2563-4339-bb39-79371e02aa30 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:13.150357 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff65a2-2563-4339-bb39-79371e02aa30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:13.150661 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff65a2-2563-4339-bb39-79371e02aa30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:13.152067 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff65a2-2563-4339-bb39-79371e02aa30 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:13.165803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 432/882] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:13.172035 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca25508-04d8-4068-a3d8-8c2f824d92d6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:13.225697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:13.287538 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca25508-04d8-4068-a3d8-8c2f824d92d6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:13.328508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967642c-04db-459d-b0ed-ab63abef7dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:13.329276 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967642c-04db-459d-b0ed-ab63abef7dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:13.329584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967642c-04db-459d-b0ed-ab63abef7dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:13.329832 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967642c-04db-459d-b0ed-ab63abef7dc1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:13.339711 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967642c-04db-459d-b0ed-ab63abef7dc1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:13.341283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967642c-04db-459d-b0ed-ab63abef7dc1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:13.353950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-5967642c-04db-459d-b0ed-ab63abef7dc1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:21:13.353950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:13.353950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:13.353950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:13.353950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:13.353950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:13.353950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:13.353950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:13.353950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:13.353950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:13.353950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:13.353950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:13.353950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:21:13.353950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:21:13.353950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:21:13.353950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:21:13.355552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:13.355552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:13.355552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:21:13.355552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:13.355552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:21:13.355552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:13.355552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:13.355552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:13.355552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:13.355552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:13.355552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:13.355552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:13.355552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:13.355552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:13.355552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:13.355552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:13.355552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:13.355552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:13.357081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:13.357081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:13.357081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:21:13.357081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:21:13.357081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:21:13.357081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:21:13.357081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:21:13.357081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:21:13.357081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:21:13.357081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:13.357081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 451/883] 10.222.0.83 () {56 vars in 1117 bytes} [Mon Aug 1 15:21:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:13.361408 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff65a2-2563-4339-bb39-79371e02aa30 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:13.375221 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5546b42-7fe1-43c9-bd54-0791e0be451e None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:13.375606 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5546b42-7fe1-43c9-bd54-0791e0be451e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:13.375831 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5546b42-7fe1-43c9-bd54-0791e0be451e None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:13.377409 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 452/884] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:13.383127 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f20c72-f22e-4ba0-ba15-4624bcd245f7 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:13.383971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f20c72-f22e-4ba0-ba15-4624bcd245f7 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:13.384210 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f20c72-f22e-4ba0-ba15-4624bcd245f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:13.406344 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c7f20c72-f22e-4ba0-ba15-4624bcd245f7 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:13.407476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c7f20c72-f22e-4ba0-ba15-4624bcd245f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:13.488787 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:13.498473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff65a2-2563-4339-bb39-79371e02aa30 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:13.525338 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 453/885] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:13.544266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c2d9b-71c1-48fa-84a7-6077fe2466a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:13.545027 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c2d9b-71c1-48fa-84a7-6077fe2466a4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:13.545340 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c2d9b-71c1-48fa-84a7-6077fe2466a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:13.545714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c2d9b-71c1-48fa-84a7-6077fe2466a4 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:13.546603 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c2d9b-71c1-48fa-84a7-6077fe2466a4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:13.546974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c2d9b-71c1-48fa-84a7-6077fe2466a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:13.548509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c2d9b-71c1-48fa-84a7-6077fe2466a4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:13.566891 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f20c72-f22e-4ba0-ba15-4624bcd245f7 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:13.572682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 433/886] 10.222.0.83 () {56 vars in 1149 bytes} [Mon Aug 1 15:21:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:13.578720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c2d9b-71c1-48fa-84a7-6077fe2466a4 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:13.634812 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:13.699203 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c2d9b-71c1-48fa-84a7-6077fe2466a4 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:13.741005 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203d588-63d7-4061-b0ae-4144a1235fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:13.741623 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203d588-63d7-4061-b0ae-4144a1235fec None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:13.741960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203d588-63d7-4061-b0ae-4144a1235fec None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:13.742257 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203d588-63d7-4061-b0ae-4144a1235fec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:13.748908 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f20c72-f22e-4ba0-ba15-4624bcd245f7 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:13.786429 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:13.793713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81fe101-1ea9-484a-9ebc-50c533623ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:13.794634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81fe101-1ea9-484a-9ebc-50c533623ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:13.795104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81fe101-1ea9-484a-9ebc-50c533623ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:13.795540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81fe101-1ea9-484a-9ebc-50c533623ae5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:13.796722 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81fe101-1ea9-484a-9ebc-50c533623ae5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:13.799660 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81fe101-1ea9-484a-9ebc-50c533623ae5 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:13.813768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-a81fe101-1ea9-484a-9ebc-50c533623ae5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:21:13.813768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:13.813768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:21:13.813768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:21:13.813768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:21:13.813768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:21:13.813768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:21:13.813768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:13.813768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:21:13.813768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:13.813768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:13.813768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:13.813768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:13.813768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:13.813768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:13.813768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:13.813768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:13.814918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:13.814918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:13.814918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:13.814918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:13.814918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:13.814918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:13.814918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:21:13.814918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:21:13.814918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:21:13.814918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:21:13.814918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:13.814918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:13.814918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:13.814918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:13.814918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:13.814918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:13.815794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:13.815794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:13.815794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:13.815794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:13.815794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:13.815794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:13.815794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:13.815794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:13.815794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:13.815794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:13.815794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:21:13.815794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:13.815794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:21:13.815794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:13.825119 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 454/887] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:21:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:13.826805 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81fe101-1ea9-484a-9ebc-50c533623ae5 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:13.828418 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203d588-63d7-4061-b0ae-4144a1235fec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:13.830462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203d588-63d7-4061-b0ae-4144a1235fec None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:13.950754 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:13.956502 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81fe101-1ea9-484a-9ebc-50c533623ae5 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:13.976144 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 434/888] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:21:13 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:21:13.976144 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d228477-b29d-48d6-8c41-d61502435276 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:13.976144 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d228477-b29d-48d6-8c41-d61502435276 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:13.976528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d228477-b29d-48d6-8c41-d61502435276 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:13.978654 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 435/889] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:13.981947 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d228477-b29d-48d6-8c41-d61502435276 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:14.010444 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e4f6b-71cb-4478-aaf9-0687d870cd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:14.011244 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e4f6b-71cb-4478-aaf9-0687d870cd5c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:14.011634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e4f6b-71cb-4478-aaf9-0687d870cd5c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:14.012010 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e4f6b-71cb-4478-aaf9-0687d870cd5c None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:14.013042 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e4f6b-71cb-4478-aaf9-0687d870cd5c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:14.013208 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e4f6b-71cb-4478-aaf9-0687d870cd5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:14.014709 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e4f6b-71cb-4478-aaf9-0687d870cd5c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:14.024898 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 455/890] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:21:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:14.026314 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f922b-f6e5-4793-9fa3-163988ca3b09 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:14.026602 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f922b-f6e5-4793-9fa3-163988ca3b09 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:14.026874 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f922b-f6e5-4793-9fa3-163988ca3b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:14.046233 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-fe4f922b-f6e5-4793-9fa3-163988ca3b09 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:14.047466 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fe4f922b-f6e5-4793-9fa3-163988ca3b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:14.048580 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:14.136011 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d228477-b29d-48d6-8c41-d61502435276 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:14.165951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 456/891] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:14.165951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f922b-f6e5-4793-9fa3-163988ca3b09 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:14.204901 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4a062-0fac-451a-8362-7264e10fc063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:14.205568 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4a062-0fac-451a-8362-7264e10fc063 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:14.205967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4a062-0fac-451a-8362-7264e10fc063 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:14.206375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4a062-0fac-451a-8362-7264e10fc063 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:14.220352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4a062-0fac-451a-8362-7264e10fc063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:14.223008 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4a062-0fac-451a-8362-7264e10fc063 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:14.244781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-a7c4a062-0fac-451a-8362-7264e10fc063 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:14.244781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:14.244781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:14.244781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:14.244781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:14.244781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:14.244781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:14.244781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:14.244781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:14.244781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:14.244781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:14.244781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:14.244781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:14.244781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:14.244781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:14.246761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:14.246761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:14.246761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:14.246761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:14.246761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:14.246761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:14.246761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:14.246761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:14.246761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:14.246761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:14.246761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:14.246761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:14.246761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:14.246761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:14.246761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:14.246761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:14.246761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:14.248434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:14.248434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:14.248434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:14.248434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:14.248434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 436/892] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:14.248948 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4a062-0fac-451a-8362-7264e10fc063 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:14.296343 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:14.309577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f922b-f6e5-4793-9fa3-163988ca3b09 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:14.356103 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9f208-23df-430a-b584-cd51ea76ecfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:14.356744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9f208-23df-430a-b584-cd51ea76ecfa None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:14.357027 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9f208-23df-430a-b584-cd51ea76ecfa None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:14.357298 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9f208-23df-430a-b584-cd51ea76ecfa None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:14.358316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9f208-23df-430a-b584-cd51ea76ecfa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:14.358624 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9f208-23df-430a-b584-cd51ea76ecfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:14.361023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9f208-23df-430a-b584-cd51ea76ecfa None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:14.366514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 457/893] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:21:14 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:14.367420 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9f208-23df-430a-b584-cd51ea76ecfa None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:14.401476 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4a062-0fac-451a-8362-7264e10fc063 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:14.404890 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:14.456295 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cef29-7494-4168-8cc0-a6f55bd78a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:14.457041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cef29-7494-4168-8cc0-a6f55bd78a7a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:14.457327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cef29-7494-4168-8cc0-a6f55bd78a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:14.457629 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cef29-7494-4168-8cc0-a6f55bd78a7a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:14.458309 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cef29-7494-4168-8cc0-a6f55bd78a7a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:14.459906 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cef29-7494-4168-8cc0-a6f55bd78a7a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:14.472079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-c70cef29-7494-4168-8cc0-a6f55bd78a7a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:21:14.472079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:14.472079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:21:14.472079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:21:14.472079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:21:14.472079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:21:14.472079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:21:14.472079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:14.472079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:21:14.472079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:14.472079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:14.472079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:14.472079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:14.472079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:14.472079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:14.472079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:14.472079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:14.473258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:14.473258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:14.473258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:14.473258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:14.473258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:14.473258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:14.473258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:21:14.473258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:21:14.473258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:21:14.473258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:21:14.473258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:14.473258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:14.473258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:14.473258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:14.473258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:14.473258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:14.474185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:14.474185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:14.474185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:14.474185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:14.474185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:14.474185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:14.474185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:14.474185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:14.474185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:14.474185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:14.474185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:21:14.474185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:14.474185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:21:14.474185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:14.474185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 437/894] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Aug 1 15:21:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:14.475214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cef29-7494-4168-8cc0-a6f55bd78a7a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:14.511811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9f208-23df-430a-b584-cd51ea76ecfa None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:14.559729 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e393696-b1cf-4d1d-898c-c4cdd20863fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:14.560366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e393696-b1cf-4d1d-898c-c4cdd20863fd None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:14.560647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e393696-b1cf-4d1d-898c-c4cdd20863fd None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:14.560918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e393696-b1cf-4d1d-898c-c4cdd20863fd None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:14.561615 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e393696-b1cf-4d1d-898c-c4cdd20863fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:14.561947 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e393696-b1cf-4d1d-898c-c4cdd20863fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:14.563324 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e393696-b1cf-4d1d-898c-c4cdd20863fd None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:14.582076 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 458/895] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:14.582076 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e393696-b1cf-4d1d-898c-c4cdd20863fd None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:14.608309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:14.642040 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cef29-7494-4168-8cc0-a6f55bd78a7a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:14.683002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919a8e4-5602-4430-a082-d753a9795f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:14.683891 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919a8e4-5602-4430-a082-d753a9795f53 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:14.684178 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919a8e4-5602-4430-a082-d753a9795f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:14.684454 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919a8e4-5602-4430-a082-d753a9795f53 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:14.685328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919a8e4-5602-4430-a082-d753a9795f53 None admin] RBAC: Adding query filter params (interface=public, service_id=9f0f07c60bb6433e85ef90cbe941af81, region_id=RegionOne) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:14.685641 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919a8e4-5602-4430-a082-d753a9795f53 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:14.687072 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919a8e4-5602-4430-a082-d753a9795f53 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:14.693152 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 438/896] 10.222.0.83 () {56 vars in 1251 bytes} [Mon Aug 1 15:21:14 2022] GET /identity/v3/endpoints?service_id=9f0f07c60bb6433e85ef90cbe941af81&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:14.693768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919a8e4-5602-4430-a082-d753a9795f53 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:14.732662 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:14.741360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e393696-b1cf-4d1d-898c-c4cdd20863fd None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:14.779486 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3c0f8-119d-4575-a0b7-86b32a9642b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:14.780258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3c0f8-119d-4575-a0b7-86b32a9642b7 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:14.780544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3c0f8-119d-4575-a0b7-86b32a9642b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:14.781655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3c0f8-119d-4575-a0b7-86b32a9642b7 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:14.782388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3c0f8-119d-4575-a0b7-86b32a9642b7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:14.782699 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3c0f8-119d-4575-a0b7-86b32a9642b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:14.784083 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3c0f8-119d-4575-a0b7-86b32a9642b7 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:14.795109 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 459/897] 10.222.0.83 () {56 vars in 1122 bytes} [Mon Aug 1 15:21:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:14.795109 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3c0f8-119d-4575-a0b7-86b32a9642b7 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:14.814945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919a8e4-5602-4430-a082-d753a9795f53 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:14.856415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb62a530-96ce-4ce9-abb7-df4ea68baa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:14.857124 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62a530-96ce-4ce9-abb7-df4ea68baa20 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:14.857414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62a530-96ce-4ce9-abb7-df4ea68baa20 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:14.857729 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62a530-96ce-4ce9-abb7-df4ea68baa20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:14.865767 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb62a530-96ce-4ce9-abb7-df4ea68baa20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:14.867933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb62a530-96ce-4ce9-abb7-df4ea68baa20 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:14.880963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-cb62a530-96ce-4ce9-abb7-df4ea68baa20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:14.880963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:14.880963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:14.880963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:14.880963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:14.880963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:14.880963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:14.880963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:14.880963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:14.880963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:14.880963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:14.880963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:14.880963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:21:14.880963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:21:14.880963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:21:14.882020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:21:14.882020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:14.882020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:14.882020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:14.882020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:14.882020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:14.882020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:14.882020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:14.882020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:14.882020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:14.882020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:14.882020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:14.882020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:14.882020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:14.882020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:14.882020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:14.882020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:14.883006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:21:14.883006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:21:14.883006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:21:14.883006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:21:14.883006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:21:14.883006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:21:14.883006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:14.883006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:14.883006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 439/898] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:21:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:14.883006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb62a530-96ce-4ce9-abb7-df4ea68baa20 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:14.899608 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:14.921739 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3c0f8-119d-4575-a0b7-86b32a9642b7 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:14.961433 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d058f0-96d3-4951-a843-560ae29420ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:14.962540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d058f0-96d3-4951-a843-560ae29420ea None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:14.962825 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d058f0-96d3-4951-a843-560ae29420ea None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:14.963101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d058f0-96d3-4951-a843-560ae29420ea None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:14.963760 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d058f0-96d3-4951-a843-560ae29420ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:14.965280 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d058f0-96d3-4951-a843-560ae29420ea None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:14.971233 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 460/899] 10.222.0.83 () {56 vars in 1087 bytes} [Mon Aug 1 15:21:14 2022] GET /identity/v3/services => generated 1268 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:21:14.972128 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d058f0-96d3-4951-a843-560ae29420ea None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:15.014985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:15.032846 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb62a530-96ce-4ce9-abb7-df4ea68baa20 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:15.084569 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ec087-156b-474c-ac37-f37b8502781e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:15.086008 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ec087-156b-474c-ac37-f37b8502781e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:15.087351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ec087-156b-474c-ac37-f37b8502781e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:15.087842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ec087-156b-474c-ac37-f37b8502781e None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:15.089029 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ec087-156b-474c-ac37-f37b8502781e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:15.089542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ec087-156b-474c-ac37-f37b8502781e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:15.092588 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ec087-156b-474c-ac37-f37b8502781e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:15.098964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 440/900] 10.222.0.83 () {56 vars in 1122 bytes} [Mon Aug 1 15:21:14 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:15.137830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d058f0-96d3-4951-a843-560ae29420ea None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:15.184622 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1cb71-7d9e-451c-9a27-8f742c80aa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:15.185276 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1cb71-7d9e-451c-9a27-8f742c80aa19 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:15.185753 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1cb71-7d9e-451c-9a27-8f742c80aa19 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:15.186242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1cb71-7d9e-451c-9a27-8f742c80aa19 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:15.187028 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1cb71-7d9e-451c-9a27-8f742c80aa19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:15.187338 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1cb71-7d9e-451c-9a27-8f742c80aa19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:15.189048 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1cb71-7d9e-451c-9a27-8f742c80aa19 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:15.205133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 461/901] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:21:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:15.214285 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ec087-156b-474c-ac37-f37b8502781e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:15.314890 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80130b48-f748-4265-af6b-8c25964480ab None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:15.315380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80130b48-f748-4265-af6b-8c25964480ab None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:15.315380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80130b48-f748-4265-af6b-8c25964480ab None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:15.317583 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 462/902] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:15.323781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1014dd-ca97-4725-bcb6-bd36a95386f4 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:15.324156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1014dd-ca97-4725-bcb6-bd36a95386f4 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:15.324457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1014dd-ca97-4725-bcb6-bd36a95386f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:15.335702 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:15.339945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ec087-156b-474c-ac37-f37b8502781e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:15.349303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2e1014dd-ca97-4725-bcb6-bd36a95386f4 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:15.351336 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2e1014dd-ca97-4725-bcb6-bd36a95386f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:15.387609 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c45890-7e90-4211-a62f-6bbb75858de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:15.388532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c45890-7e90-4211-a62f-6bbb75858de0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:15.388829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c45890-7e90-4211-a62f-6bbb75858de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:15.389113 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c45890-7e90-4211-a62f-6bbb75858de0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:15.390728 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c45890-7e90-4211-a62f-6bbb75858de0 None admin] RBAC: Adding query filter params (role.id=37a24cea29194c7ca77701607e008914, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=f36c6987ef774a1684c55f885e2a4f67) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:15.391190 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c45890-7e90-4211-a62f-6bbb75858de0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:15.393673 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c45890-7e90-4211-a62f-6bbb75858de0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:15.402045 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 441/903] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:21:15 2022] GET /identity/v3/role_assignments?user.id=f36c6987ef774a1684c55f885e2a4f67&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=37a24cea29194c7ca77701607e008914 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:15.442561 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 463/904] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:15 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:21:15.467116 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c45890-7e90-4211-a62f-6bbb75858de0 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:15.504833 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:15.596494 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c45890-7e90-4211-a62f-6bbb75858de0 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:15.638921 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f4825-618e-4336-9c31-239e5b25268b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:15.640010 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f4825-618e-4336-9c31-239e5b25268b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:15.640298 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f4825-618e-4336-9c31-239e5b25268b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:15.640576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f4825-618e-4336-9c31-239e5b25268b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:15.641215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f4825-618e-4336-9c31-239e5b25268b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:15.642661 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f4825-618e-4336-9c31-239e5b25268b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:15.653905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-489f4825-618e-4336-9c31-239e5b25268b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:21:15.653905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:15.653905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:21:15.653905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:21:15.653905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:21:15.653905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:21:15.653905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:21:15.653905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:15.653905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:21:15.653905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:15.653905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:15.653905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:15.653905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:15.653905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:15.653905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:15.653905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:15.654900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:15.654900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:15.654900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:15.654900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:15.654900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:15.654900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:15.654900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:15.654900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:21:15.654900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:21:15.654900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:21:15.654900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:21:15.654900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:15.654900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:15.654900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:15.654900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:15.654900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:15.655778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:15.655778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:15.655778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:15.655778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:15.655778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:15.655778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:15.655778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:15.655778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:15.655778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:15.655778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:15.655778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:15.655778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:21:15.655778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:15.655778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:21:15.655778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:15.655778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 442/905] 10.222.0.83 () {56 vars in 1132 bytes} [Mon Aug 1 15:21:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:15.663829 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1014dd-ca97-4725-bcb6-bd36a95386f4 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:15.780737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1014dd-ca97-4725-bcb6-bd36a95386f4 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:15.820928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9110c46-ab3b-4236-bd2a-f5a9bfeff3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:15.822056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9110c46-ab3b-4236-bd2a-f5a9bfeff3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:15.822556 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9110c46-ab3b-4236-bd2a-f5a9bfeff3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:15.823020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9110c46-ab3b-4236-bd2a-f5a9bfeff3e4 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:15.823908 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9110c46-ab3b-4236-bd2a-f5a9bfeff3e4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:15.824438 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9110c46-ab3b-4236-bd2a-f5a9bfeff3e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:15.826145 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9110c46-ab3b-4236-bd2a-f5a9bfeff3e4 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:15.827582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:15.833653 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 464/906] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Aug 1 15:21:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:15.840798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f4825-618e-4336-9c31-239e5b25268b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:15.943889 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:15.963095 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f4825-618e-4336-9c31-239e5b25268b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:16.003343 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf068f-e08f-4ab5-a57c-7e1d95bf967b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:16.004641 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf068f-e08f-4ab5-a57c-7e1d95bf967b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:16.005129 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf068f-e08f-4ab5-a57c-7e1d95bf967b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:16.005620 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf068f-e08f-4ab5-a57c-7e1d95bf967b None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:16.006555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf068f-e08f-4ab5-a57c-7e1d95bf967b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:16.007322 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf068f-e08f-4ab5-a57c-7e1d95bf967b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:16.009547 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf068f-e08f-4ab5-a57c-7e1d95bf967b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:16.015805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 443/907] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Aug 1 15:21:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:16.021955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9110c46-ab3b-4236-bd2a-f5a9bfeff3e4 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:16.131498 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9110c46-ab3b-4236-bd2a-f5a9bfeff3e4 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:16.170105 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37729521-2b53-4a57-b74f-76f7540a0cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:16.171114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37729521-2b53-4a57-b74f-76f7540a0cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:16.171654 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37729521-2b53-4a57-b74f-76f7540a0cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:16.172125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37729521-2b53-4a57-b74f-76f7540a0cc0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:16.173113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37729521-2b53-4a57-b74f-76f7540a0cc0 None admin] RBAC: Adding query filter params (interface=public, service_id=47d336414834471faca4668648679264, region_id=RegionOne) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:16.173632 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37729521-2b53-4a57-b74f-76f7540a0cc0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:16.175367 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37729521-2b53-4a57-b74f-76f7540a0cc0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:16.182176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 465/908] 10.222.0.83 () {56 vars in 1251 bytes} [Mon Aug 1 15:21:16 2022] GET /identity/v3/endpoints?service_id=47d336414834471faca4668648679264&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:16.188518 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf068f-e08f-4ab5-a57c-7e1d95bf967b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:16.300048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf068f-e08f-4ab5-a57c-7e1d95bf967b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:16.341878 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91141c-4cf1-40ff-9d6b-806d4697fdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:16.342922 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91141c-4cf1-40ff-9d6b-806d4697fdac None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:16.343496 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91141c-4cf1-40ff-9d6b-806d4697fdac None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:16.343978 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91141c-4cf1-40ff-9d6b-806d4697fdac None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:16.344841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d91141c-4cf1-40ff-9d6b-806d4697fdac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:16.346472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d91141c-4cf1-40ff-9d6b-806d4697fdac None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:16.353544 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 444/909] 10.222.0.83 () {56 vars in 1087 bytes} [Mon Aug 1 15:21:16 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:21:17.546404 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e36bc-2c84-4d75-8df8-ea6e2af27c98 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:17.547671 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e36bc-2c84-4d75-8df8-ea6e2af27c98 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:17.548091 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3eb497-9f15-4543-9b21-3e9b14419e03 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:17.548537 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e36bc-2c84-4d75-8df8-ea6e2af27c98 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:17.548618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3eb497-9f15-4543-9b21-3e9b14419e03 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:17.548618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3eb497-9f15-4543-9b21-3e9b14419e03 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:17.550150 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 445/910] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:21:17.550608 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 466/911] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:17.556106 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2096e7f-5c43-4e46-afa4-d6b9de3e014f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:17.556390 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2096e7f-5c43-4e46-afa4-d6b9de3e014f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:17.556663 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2096e7f-5c43-4e46-afa4-d6b9de3e014f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:17.558960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f3119-94f2-4a2c-a4b1-975803f1a6ff None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:17.559506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f3119-94f2-4a2c-a4b1-975803f1a6ff None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:17.560061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f3119-94f2-4a2c-a4b1-975803f1a6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:17.577598 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c2096e7f-5c43-4e46-afa4-d6b9de3e014f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:17.578929 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c2096e7f-5c43-4e46-afa4-d6b9de3e014f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:17.581653 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e35f3119-94f2-4a2c-a4b1-975803f1a6ff None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:17.583097 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e35f3119-94f2-4a2c-a4b1-975803f1a6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:17.588066 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:17.653582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:17.665939 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 467/912] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:17 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:21:17.674461 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 446/913] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:17 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:17.697103 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f3119-94f2-4a2c-a4b1-975803f1a6ff None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:17.698908 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2096e7f-5c43-4e46-afa4-d6b9de3e014f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:17.822752 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f3119-94f2-4a2c-a4b1-975803f1a6ff None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:17.834578 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2096e7f-5c43-4e46-afa4-d6b9de3e014f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:17.870342 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf95805-47dc-44e1-bbe2-09764e40da81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:17.871328 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf95805-47dc-44e1-bbe2-09764e40da81 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:17.871815 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf95805-47dc-44e1-bbe2-09764e40da81 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:17.872324 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf95805-47dc-44e1-bbe2-09764e40da81 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:17.873217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf95805-47dc-44e1-bbe2-09764e40da81 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:17.873443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b1994-2733-4838-9636-0deb68cd70e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:17.874450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b1994-2733-4838-9636-0deb68cd70e7 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:17.874887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b1994-2733-4838-9636-0deb68cd70e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:17.875307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b1994-2733-4838-9636-0deb68cd70e7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:17.875756 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf95805-47dc-44e1-bbe2-09764e40da81 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:17.876266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b1994-2733-4838-9636-0deb68cd70e7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:17.878480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b1994-2733-4838-9636-0deb68cd70e7 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:17.894829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-f82b1994-2733-4838-9636-0deb68cd70e7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:21:17.894829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:17.894829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:21:17.894829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:21:17.894829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:21:17.894829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:21:17.894829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:21:17.894829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:17.894829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:21:17.894829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:17.894829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:17.894829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:17.894829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:17.894829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:17.894829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:17.894829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:17.894829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:17.902823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:17.902823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:17.902823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:17.902823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:17.902823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:17.902823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:17.902823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:21:17.902823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:21:17.902823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:21:17.902823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:21:17.902823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:17.902823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:17.902823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:17.902823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:17.902823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:17.902823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:17.904183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:17.904183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:17.904183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:17.904183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:17.904183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:17.904183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:17.904183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:17.904183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:17.904183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:17.904183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:17.904183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:21:17.904183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:17.904183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:21:17.904183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:17.906591 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 468/914] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Aug 1 15:21:17 2022] POST /identity/v3/services => generated 241 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:21:17.906591 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1022eff-d654-4813-b82f-4aeeda3c8ab1 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:17.906591 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1022eff-d654-4813-b82f-4aeeda3c8ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:17.906591 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1022eff-d654-4813-b82f-4aeeda3c8ab1 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:17.906591 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 469/915] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:17.907287 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 447/916] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:21:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:17.907430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1022eff-d654-4813-b82f-4aeeda3c8ab1 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:17.912481 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d2ce6-9ea1-4b5d-8b93-041da95d75b6 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:17.912481 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d2ce6-9ea1-4b5d-8b93-041da95d75b6 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:17.912481 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d2ce6-9ea1-4b5d-8b93-041da95d75b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:17.932802 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:17.937725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6a3d2ce6-9ea1-4b5d-8b93-041da95d75b6 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:17.939587 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6a3d2ce6-9ea1-4b5d-8b93-041da95d75b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:17.989554 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:18.058385 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1022eff-d654-4813-b82f-4aeeda3c8ab1 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:18.072001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 448/917] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:21:18.095143 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d2ce6-9ea1-4b5d-8b93-041da95d75b6 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:18.112741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56816a-ef5a-4edc-bfd7-5143318cf2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:18.113369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56816a-ef5a-4edc-bfd7-5143318cf2cc None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:18.113671 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56816a-ef5a-4edc-bfd7-5143318cf2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:18.114059 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56816a-ef5a-4edc-bfd7-5143318cf2cc None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:18.114750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d56816a-ef5a-4edc-bfd7-5143318cf2cc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:18.115054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d56816a-ef5a-4edc-bfd7-5143318cf2cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:18.116447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d56816a-ef5a-4edc-bfd7-5143318cf2cc None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:18.125242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 470/918] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:21:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:18.131765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56816a-ef5a-4edc-bfd7-5143318cf2cc None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:18.230153 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d2ce6-9ea1-4b5d-8b93-041da95d75b6 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:18.263519 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56816a-ef5a-4edc-bfd7-5143318cf2cc None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:18.268703 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3198953-abdc-4cac-9825-26398407ec3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:18.269456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3198953-abdc-4cac-9825-26398407ec3f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:18.269715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3198953-abdc-4cac-9825-26398407ec3f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:18.270033 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3198953-abdc-4cac-9825-26398407ec3f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:18.273026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:18.281233 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:18.283158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3198953-abdc-4cac-9825-26398407ec3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:21:18.288319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-a3198953-abdc-4cac-9825-26398407ec3f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:21:18.288319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:18.288319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:18.288319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:18.288319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:18.288319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:18.288319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:18.288319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:18.288319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:18.288319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:18.288319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:18.288319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:18.288319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:21:18.288319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:21:18.288319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:21:18.288319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:21:18.289812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:21:18.289812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:21:18.289812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:21:18.289812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:21:18.289812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:18.289812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:18.289812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:18.289812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:18.289812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:18.289812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:18.289812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:18.289812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:18.289812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:18.289812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:18.289812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:18.289812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:18.291166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:18.291166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:18.291166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:18.291166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:18.291166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:21:18.291166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:21:18.291166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:21:18.291166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:21:18.291166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:21:18.291166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:21:18.291166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:21:18.291166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:18.294006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 449/919] 10.222.0.83 () {56 vars in 1120 bytes} [Mon Aug 1 15:21:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:21:18.296326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3198953-abdc-4cac-9825-26398407ec3f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:18.313088 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938dcb38-6232-4602-b1c3-21b2a086f83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:18.314246 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dcb38-6232-4602-b1c3-21b2a086f83b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:18.314850 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dcb38-6232-4602-b1c3-21b2a086f83b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:18.315319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dcb38-6232-4602-b1c3-21b2a086f83b None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:18.316285 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938dcb38-6232-4602-b1c3-21b2a086f83b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:18.316813 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938dcb38-6232-4602-b1c3-21b2a086f83b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:18.318489 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938dcb38-6232-4602-b1c3-21b2a086f83b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:18.329935 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 471/920] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Aug 1 15:21:18 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:18.332042 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938dcb38-6232-4602-b1c3-21b2a086f83b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:18.408832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3198953-abdc-4cac-9825-26398407ec3f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:18.450443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206568c-abd0-4e4f-bf71-8a3e882b4c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:18.451003 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206568c-abd0-4e4f-bf71-8a3e882b4c62 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:18.451355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206568c-abd0-4e4f-bf71-8a3e882b4c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:18.451703 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206568c-abd0-4e4f-bf71-8a3e882b4c62 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:18.452503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206568c-abd0-4e4f-bf71-8a3e882b4c62 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:18.452934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206568c-abd0-4e4f-bf71-8a3e882b4c62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:18.454483 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206568c-abd0-4e4f-bf71-8a3e882b4c62 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:18.465055 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938dcb38-6232-4602-b1c3-21b2a086f83b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:18.469737 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 450/921] 10.222.0.83 () {56 vars in 1115 bytes} [Mon Aug 1 15:21:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:18.470775 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206568c-abd0-4e4f-bf71-8a3e882b4c62 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:18.509268 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216dccb5-6824-4e6e-931d-61cb602640ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:18.510403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216dccb5-6824-4e6e-931d-61cb602640ba None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:18.510947 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216dccb5-6824-4e6e-931d-61cb602640ba None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:18.511417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216dccb5-6824-4e6e-931d-61cb602640ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:18.512339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216dccb5-6824-4e6e-931d-61cb602640ba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:18.515296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216dccb5-6824-4e6e-931d-61cb602640ba None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:18.540907 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:18.550307 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 472/922] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:21:18 2022] POST /identity/v3/endpoints => generated 346 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:21:18.594198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206568c-abd0-4e4f-bf71-8a3e882b4c62 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:18.642260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae41b9-3c79-41b0-8251-cfc043c2a942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:18.642921 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae41b9-3c79-41b0-8251-cfc043c2a942 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:18.643202 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae41b9-3c79-41b0-8251-cfc043c2a942 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:18.643476 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae41b9-3c79-41b0-8251-cfc043c2a942 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:18.657544 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae41b9-3c79-41b0-8251-cfc043c2a942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:18.659269 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae41b9-3c79-41b0-8251-cfc043c2a942 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:18.677308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-7cae41b9-3c79-41b0-8251-cfc043c2a942 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:18.677308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:18.677308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:18.677308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:18.677308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:18.677308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:18.677308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:18.677308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:18.677308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:18.677308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:18.677308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:18.677308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:18.677308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:18.677308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:18.677308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:18.678595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:18.678595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:18.678595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:18.678595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:18.678595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:18.678595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:18.678595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:18.678595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:18.678595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:18.678595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:18.678595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:18.678595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:18.678595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:18.678595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:18.678595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:18.678595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:18.678595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:18.679633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:18.679633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:18.679633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:18.679633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:18.679633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 451/923] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:18.687265 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216dccb5-6824-4e6e-931d-61cb602640ba None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:18.728564 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:18.853575 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216dccb5-6824-4e6e-931d-61cb602640ba None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:18.896798 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5cc7e-bc94-4195-90d3-2176625bffed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:18.897508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5cc7e-bc94-4195-90d3-2176625bffed None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:18.898009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5cc7e-bc94-4195-90d3-2176625bffed None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:18.898337 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5cc7e-bc94-4195-90d3-2176625bffed None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:18.899717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5cc7e-bc94-4195-90d3-2176625bffed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:18.900066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5cc7e-bc94-4195-90d3-2176625bffed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:18.901498 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5cc7e-bc94-4195-90d3-2176625bffed None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:18.917834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 473/924] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:18.923779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae41b9-3c79-41b0-8251-cfc043c2a942 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:19.060480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae41b9-3c79-41b0-8251-cfc043c2a942 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:19.073741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:19.118777 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fc233-da12-4a30-8b65-b7c913a61f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:19.119743 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fc233-da12-4a30-8b65-b7c913a61f2f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:19.120170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fc233-da12-4a30-8b65-b7c913a61f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:19.120601 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fc233-da12-4a30-8b65-b7c913a61f2f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:19.136876 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268fc233-da12-4a30-8b65-b7c913a61f2f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:19.139240 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268fc233-da12-4a30-8b65-b7c913a61f2f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:19.160527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-268fc233-da12-4a30-8b65-b7c913a61f2f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:21:19.160527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:19.160527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:19.160527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:19.160527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:19.160527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:19.160527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:19.160527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:19.160527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:19.160527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:19.160527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:19.160527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:19.160527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:21:19.160527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:21:19.160527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:21:19.160527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:21:19.161996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:19.161996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:19.161996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:21:19.161996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:19.161996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:21:19.161996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:19.161996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:19.161996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:19.161996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:19.161996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:19.161996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:19.161996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:19.161996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:19.161996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:19.161996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:19.161996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:19.161996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:19.161996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:19.163225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:19.163225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:19.163225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:21:19.163225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:21:19.163225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:21:19.163225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:21:19.163225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:21:19.163225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:21:19.163225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:21:19.163225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:19.163225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 452/925] 10.222.0.83 () {56 vars in 1117 bytes} [Mon Aug 1 15:21:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:19.168977 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5cc7e-bc94-4195-90d3-2176625bffed None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:19.294423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:19.304629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5cc7e-bc94-4195-90d3-2176625bffed None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:19.344611 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe4846-b456-483f-8222-4d87a72e66f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:19.345245 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe4846-b456-483f-8222-4d87a72e66f5 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:19.345525 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe4846-b456-483f-8222-4d87a72e66f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:19.345863 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe4846-b456-483f-8222-4d87a72e66f5 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:19.346831 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe4846-b456-483f-8222-4d87a72e66f5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:19.347137 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe4846-b456-483f-8222-4d87a72e66f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:19.348665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe4846-b456-483f-8222-4d87a72e66f5 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:19.371405 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 474/926] 10.222.0.83 () {56 vars in 1149 bytes} [Mon Aug 1 15:21:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:19.377182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fc233-da12-4a30-8b65-b7c913a61f2f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:19.392179 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db4845-2eca-4efb-8ec5-84585434cc91 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:19.392464 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db4845-2eca-4efb-8ec5-84585434cc91 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:19.392749 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db4845-2eca-4efb-8ec5-84585434cc91 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:19.394380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 475/927] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:21:19.401343 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588a21a-8754-4f01-8fd7-f91392ff130f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:19.401626 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588a21a-8754-4f01-8fd7-f91392ff130f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:19.402080 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588a21a-8754-4f01-8fd7-f91392ff130f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:19.426825 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-6588a21a-8754-4f01-8fd7-f91392ff130f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:19.429182 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6588a21a-8754-4f01-8fd7-f91392ff130f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:19.499495 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fc233-da12-4a30-8b65-b7c913a61f2f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:19.538138 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 476/928] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:21:19.553691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b14c1-6851-450c-8e5b-f89d6ca3e991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:19.555316 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b14c1-6851-450c-8e5b-f89d6ca3e991 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:19.555809 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b14c1-6851-450c-8e5b-f89d6ca3e991 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:19.556424 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b14c1-6851-450c-8e5b-f89d6ca3e991 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:19.560008 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588a21a-8754-4f01-8fd7-f91392ff130f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:19.576050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b14c1-6851-450c-8e5b-f89d6ca3e991 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:19.578070 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b14c1-6851-450c-8e5b-f89d6ca3e991 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:19.605974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-d91b14c1-6851-450c-8e5b-f89d6ca3e991 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:19.605974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:19.605974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:19.605974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:19.605974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:19.605974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:19.605974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:19.605974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:19.605974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:19.605974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:19.605974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:19.605974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:19.605974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:19.605974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:19.605974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:19.607555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:19.607555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:19.607555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:19.607555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:19.607555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:19.607555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:19.607555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:19.607555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:19.607555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:19.607555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:19.607555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:19.607555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:19.607555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:19.607555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:19.607555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:19.607555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:19.607555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:19.608639 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:19.608639 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:19.608639 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:19.608639 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:19.614348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 453/929] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:19.617989 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b14c1-6851-450c-8e5b-f89d6ca3e991 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:19.647145 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:19.685792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:19.722837 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588a21a-8754-4f01-8fd7-f91392ff130f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:19.763676 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b14c1-6851-450c-8e5b-f89d6ca3e991 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:19.787996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f8791-499c-436d-8e10-4d01ded734ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:19.789975 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f8791-499c-436d-8e10-4d01ded734ab None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:19.790570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f8791-499c-436d-8e10-4d01ded734ab None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:19.791108 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f8791-499c-436d-8e10-4d01ded734ab None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:19.792320 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f8791-499c-436d-8e10-4d01ded734ab None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:19.794922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f8791-499c-436d-8e10-4d01ded734ab None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:19.811938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-eb9f8791-499c-436d-8e10-4d01ded734ab None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:21:19.811938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:19.811938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:21:19.811938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:21:19.811938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:21:19.811938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:21:19.811938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:21:19.811938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:19.811938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:21:19.811938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:19.811938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:19.811938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:19.811938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:19.811938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:19.811938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:19.811938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:19.813223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:19.813223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:19.813223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:19.813223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:19.813223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:19.813223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:19.813223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:19.813223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:21:19.813223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:21:19.813223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:21:19.813223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:21:19.813223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:19.813223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:19.813223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:19.813223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:19.813223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:19.814196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:19.814196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:19.814196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:19.814196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:19.814196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:19.814196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:19.814196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:19.814196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:19.814196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:19.814196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:19.814196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:19.814196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:21:19.814196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:19.814196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:21:19.814196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:19.815009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 477/930] 10.222.0.83 () {56 vars in 1132 bytes} [Mon Aug 1 15:21:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:19.822343 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f8791-499c-436d-8e10-4d01ded734ab None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:19.834779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe1aae-c449-4cb1-be3f-a918b43ecd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:19.835783 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe1aae-c449-4cb1-be3f-a918b43ecd93 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:19.836279 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe1aae-c449-4cb1-be3f-a918b43ecd93 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:19.836740 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe1aae-c449-4cb1-be3f-a918b43ecd93 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:19.837990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe1aae-c449-4cb1-be3f-a918b43ecd93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:19.838502 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe1aae-c449-4cb1-be3f-a918b43ecd93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:19.841046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe1aae-c449-4cb1-be3f-a918b43ecd93 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:19.870532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 454/931] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:19.874924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe1aae-c449-4cb1-be3f-a918b43ecd93 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:20.007133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:20.033471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe1aae-c449-4cb1-be3f-a918b43ecd93 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:20.039076 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f8791-499c-436d-8e10-4d01ded734ab None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:20.074498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ceae22-3a4c-4bd6-b4bf-3323122993c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:20.075158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ceae22-3a4c-4bd6-b4bf-3323122993c8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:20.075443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ceae22-3a4c-4bd6-b4bf-3323122993c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:20.075708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ceae22-3a4c-4bd6-b4bf-3323122993c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:20.084216 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ceae22-3a4c-4bd6-b4bf-3323122993c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:20.086645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ceae22-3a4c-4bd6-b4bf-3323122993c8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:20.099933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-a1ceae22-3a4c-4bd6-b4bf-3323122993c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:20.099933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:20.099933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:20.099933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:20.099933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:20.099933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:20.099933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:20.099933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:20.099933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:20.099933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:20.099933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:20.099933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:20.099933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:21:20.099933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:21:20.099933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:21:20.101779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:21:20.101779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:20.101779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:20.101779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:20.101779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:20.101779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:20.101779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:20.101779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:20.101779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:20.101779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:20.101779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:20.101779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:20.101779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:20.101779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:20.101779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:20.101779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:20.101779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:20.104158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:21:20.104158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:21:20.104158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:21:20.104158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:21:20.104158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:21:20.104158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:21:20.104158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:20.104158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:20.107958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 455/932] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:21:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:21:20.109225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ceae22-3a4c-4bd6-b4bf-3323122993c8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:20.124053 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f864384-a191-435f-9da7-05dfe2f2197e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:20.126033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f864384-a191-435f-9da7-05dfe2f2197e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:20.127001 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f864384-a191-435f-9da7-05dfe2f2197e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:20.127878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f864384-a191-435f-9da7-05dfe2f2197e None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:20.129658 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f864384-a191-435f-9da7-05dfe2f2197e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:20.130637 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f864384-a191-435f-9da7-05dfe2f2197e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:20.133829 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f864384-a191-435f-9da7-05dfe2f2197e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:20.145219 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 478/933] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Aug 1 15:21:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:20.153052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f864384-a191-435f-9da7-05dfe2f2197e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:20.175955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:20.281633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ceae22-3a4c-4bd6-b4bf-3323122993c8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:20.297778 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f864384-a191-435f-9da7-05dfe2f2197e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:20.342790 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85571dda-d319-4287-8e38-fb4761151fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:20.343917 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85571dda-d319-4287-8e38-fb4761151fad None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:20.344457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85571dda-d319-4287-8e38-fb4761151fad None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:20.344988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85571dda-d319-4287-8e38-fb4761151fad None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:20.346254 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85571dda-d319-4287-8e38-fb4761151fad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:20.346884 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85571dda-d319-4287-8e38-fb4761151fad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:20.347461 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aabd755-80fc-4940-897c-fcc00a2b8e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:20.348545 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabd755-80fc-4940-897c-fcc00a2b8e17 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:20.349054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabd755-80fc-4940-897c-fcc00a2b8e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:20.349286 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85571dda-d319-4287-8e38-fb4761151fad None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:20.349567 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabd755-80fc-4940-897c-fcc00a2b8e17 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:20.350898 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aabd755-80fc-4940-897c-fcc00a2b8e17 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:20.351464 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aabd755-80fc-4940-897c-fcc00a2b8e17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:20.354693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aabd755-80fc-4940-897c-fcc00a2b8e17 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:20.359143 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:20.364842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 479/934] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Aug 1 15:21:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:20.366538 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecde2ab-aa81-4d02-ae79-8de1c659d5ff None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:20.366978 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecde2ab-aa81-4d02-ae79-8de1c659d5ff None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:20.367430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecde2ab-aa81-4d02-ae79-8de1c659d5ff None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:20.369185 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 480/935] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:20.375051 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecde2ab-aa81-4d02-ae79-8de1c659d5ff None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:20.377112 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 456/936] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:21:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:20.379117 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76932bc0-a42a-43ea-a7b0-3c781ed380ac None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:20.379563 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76932bc0-a42a-43ea-a7b0-3c781ed380ac None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:20.380058 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76932bc0-a42a-43ea-a7b0-3c781ed380ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:20.408475 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-76932bc0-a42a-43ea-a7b0-3c781ed380ac None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:20.410393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-76932bc0-a42a-43ea-a7b0-3c781ed380ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:20.497427 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:20.526136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 457/937] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:20.527652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76932bc0-a42a-43ea-a7b0-3c781ed380ac None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:20.560650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecde2ab-aa81-4d02-ae79-8de1c659d5ff None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:20.603691 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde25fc-fc01-43df-8aa6-4f43f6c42105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:20.604341 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde25fc-fc01-43df-8aa6-4f43f6c42105 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:20.604653 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde25fc-fc01-43df-8aa6-4f43f6c42105 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:20.605099 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde25fc-fc01-43df-8aa6-4f43f6c42105 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:20.605855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde25fc-fc01-43df-8aa6-4f43f6c42105 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:20.607400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde25fc-fc01-43df-8aa6-4f43f6c42105 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:20.641947 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 481/938] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:21:20 2022] POST /identity/v3/endpoints => generated 347 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:21:20.641947 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde25fc-fc01-43df-8aa6-4f43f6c42105 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:20.726634 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76932bc0-a42a-43ea-a7b0-3c781ed380ac None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:20.781261 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:20.808877 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c96a5-064b-4ee2-882b-574e79c966c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:20.810242 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c96a5-064b-4ee2-882b-574e79c966c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:20.810785 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c96a5-064b-4ee2-882b-574e79c966c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:20.811175 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c96a5-064b-4ee2-882b-574e79c966c0 None admin] PATH_INFO: `/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/f36c6987ef774a1684c55f885e2a4f67/roles/37a24cea29194c7ca77701607e008914` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:20.816338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c96a5-064b-4ee2-882b-574e79c966c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f4e27cc50bd4f9f8ced4cc544ab3064, user_id=f36c6987ef774a1684c55f885e2a4f67, role_id=37a24cea29194c7ca77701607e008914)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:20.818970 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c96a5-064b-4ee2-882b-574e79c966c0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:20.846033 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 458/939] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:21:20 2022] PUT /identity/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/f36c6987ef774a1684c55f885e2a4f67/roles/37a24cea29194c7ca77701607e008914 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:21:20.876636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde25fc-fc01-43df-8aa6-4f43f6c42105 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:20.911288 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09237b-8cc8-482b-95c8-45e03c24dbaa None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:20.911576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09237b-8cc8-482b-95c8-45e03c24dbaa None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:20.911845 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09237b-8cc8-482b-95c8-45e03c24dbaa None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:20.915503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 459/940] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:21:20.923802 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa240b-df53-4cbe-a4ca-54a69a44c53b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:20.924413 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa240b-df53-4cbe-a4ca-54a69a44c53b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:20.927256 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa240b-df53-4cbe-a4ca-54a69a44c53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:20.928643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:20.942948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d59f5c-aeed-4dfb-8bc1-a72e439909f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:20.943634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d59f5c-aeed-4dfb-8bc1-a72e439909f2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:20.943917 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d59f5c-aeed-4dfb-8bc1-a72e439909f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:20.944216 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d59f5c-aeed-4dfb-8bc1-a72e439909f2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:20.944905 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d59f5c-aeed-4dfb-8bc1-a72e439909f2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:20.946954 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d59f5c-aeed-4dfb-8bc1-a72e439909f2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:20.960146 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-41fa240b-df53-4cbe-a4ca-54a69a44c53b None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:20.960737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-29d59f5c-aeed-4dfb-8bc1-a72e439909f2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:21:20.960737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:20.960737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:21:20.960737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:21:20.960737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:21:20.960737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:21:20.960737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:21:20.960737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:20.960737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:21:20.960737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:20.960737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:20.960737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:20.960737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:20.960737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:20.960737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:20.960737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:20.960737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:20.961808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:20.961808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:20.961808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:20.961808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:20.961808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:20.961808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:20.961808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:21:20.961808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:21:20.961808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:21:20.961808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:21:20.961808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:20.961808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:20.961808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:20.961808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:20.961808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:20.961808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:20.962785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:20.962785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:20.962785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:20.962785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:20.962785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:20.962785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:20.962785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:20.962785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:20.962785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:20.962785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:20.962785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:21:20.962785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:20.962785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:21:20.962785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:20.963655 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-41fa240b-df53-4cbe-a4ca-54a69a44c53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:20.967506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 482/941] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Aug 1 15:21:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:20.979326 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d59f5c-aeed-4dfb-8bc1-a72e439909f2 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:21.054360 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 460/942] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:21.096420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa240b-df53-4cbe-a4ca-54a69a44c53b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:21.129812 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d59f5c-aeed-4dfb-8bc1-a72e439909f2 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:21.183225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09125ad6-c3d2-4e97-aa66-66ea2c463bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:21.184725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09125ad6-c3d2-4e97-aa66-66ea2c463bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:21.185122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09125ad6-c3d2-4e97-aa66-66ea2c463bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:21.185474 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09125ad6-c3d2-4e97-aa66-66ea2c463bd8 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:21.186226 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09125ad6-c3d2-4e97-aa66-66ea2c463bd8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:21.186531 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09125ad6-c3d2-4e97-aa66-66ea2c463bd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:21.188261 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09125ad6-c3d2-4e97-aa66-66ea2c463bd8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:21.197378 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 483/943] 10.222.0.83 () {56 vars in 1122 bytes} [Mon Aug 1 15:21:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:21.203507 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09125ad6-c3d2-4e97-aa66-66ea2c463bd8 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:21.241225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa240b-df53-4cbe-a4ca-54a69a44c53b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:21.263192 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:21.296593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:21.299729 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf88828-33bc-4198-b8b7-518f8556fac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:21.300605 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf88828-33bc-4198-b8b7-518f8556fac6 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:21.301147 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf88828-33bc-4198-b8b7-518f8556fac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:21.301521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf88828-33bc-4198-b8b7-518f8556fac6 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:21.302757 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf88828-33bc-4198-b8b7-518f8556fac6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:21.305463 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf88828-33bc-4198-b8b7-518f8556fac6 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:21.309318 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: WARNING keystone.server.flask.common [None req-1bf88828-33bc-4198-b8b7-518f8556fac6 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 15:21:21.333866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.sql.core [None req-1bf88828-33bc-4198-b8b7-518f8556fac6 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 15:21:21.333866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 15:21:21.333866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [parameters: {'id': 'a95321255a17439a9b37045ff247afc0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 15:21:21.333866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80349) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 15:21:21.356900 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09125ad6-c3d2-4e97-aa66-66ea2c463bd8 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:21.357111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-1bf88828-33bc-4198-b8b7-518f8556fac6 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a95321255a17439a9b37045ff247afc0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a95321255a17439a9b37045ff247afc0. Aug 01 15:21:21.357111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:21.357111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 15:21:21.357111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 15:21:21.357111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 15:21:21.357111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 15:21:21.357111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 15:21:21.357111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 15:21:21.357111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 15:21:21.357111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application conn.query(q) Aug 01 15:21:21.357111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 15:21:21.357111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 15:21:21.357111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 15:21:21.358238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application result.read() Aug 01 15:21:21.358238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 15:21:21.358238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 15:21:21.358238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 15:21:21.358238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 15:21:21.358238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 15:21:21.358238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 15:21:21.358238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 15:21:21.358238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 15:21:21.358238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 15:21:21.358238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:21.358238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 15:21:21.358238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:21.358238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:21.358238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 15:21:21.358238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 15:21:21.358238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 01 15:21:21.358238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 01 15:21:21.359312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 15:21:21.359312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application next(self.gen) Aug 01 15:21:21.359312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 15:21:21.359312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application yield resource Aug 01 15:21:21.359312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 15:21:21.359312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application next(self.gen) Aug 01 15:21:21.359312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 15:21:21.359312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 15:21:21.359312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 15:21:21.359312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 15:21:21.359312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 15:21:21.359312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise self.value Aug 01 15:21:21.359312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 15:21:21.359312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 15:21:21.359312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 15:21:21.359312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application session.commit() Aug 01 15:21:21.359312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 15:21:21.360484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 15:21:21.360484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 15:21:21.360484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 15:21:21.360484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 15:21:21.360484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self.session.flush() Aug 01 15:21:21.360484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 15:21:21.360484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 15:21:21.360484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 15:21:21.360484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 15:21:21.360484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 15:21:21.360484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application compat.raise_( Aug 01 15:21:21.360484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 15:21:21.360484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception Aug 01 15:21:21.360484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 15:21:21.360484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 15:21:21.360484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 15:21:21.360484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 15:21:21.362904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 15:21:21.362904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 15:21:21.362904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 15:21:21.362904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 15:21:21.362904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 15:21:21.362904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 15:21:21.362904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 15:21:21.362904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 15:21:21.362904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 15:21:21.362904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 15:21:21.362904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 15:21:21.362904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 15:21:21.362904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 15:21:21.362904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 15:21:21.362904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 15:21:21.365012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 15:21:21.365012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 15:21:21.365012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception Aug 01 15:21:21.365012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 15:21:21.365012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 15:21:21.365012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 15:21:21.365012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 15:21:21.365012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 15:21:21.365012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 15:21:21.365012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 15:21:21.365012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application conn.query(q) Aug 01 15:21:21.365012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 15:21:21.365012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 15:21:21.365012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 15:21:21.365012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application result.read() Aug 01 15:21:21.365012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 15:21:21.365012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 15:21:21.375652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 15:21:21.375652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 15:21:21.375652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 15:21:21.375652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 15:21:21.375652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 15:21:21.375652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 15:21:21.375652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 15:21:21.375652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 15:21:21.375652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [parameters: {'id': 'a95321255a17439a9b37045ff247afc0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 15:21:21.375652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 15:21:21.375652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:21.375652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:21.375652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:21.375652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:21.375652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 01 15:21:21.377434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 01 15:21:21.377434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 15:21:21.377434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 15:21:21.377434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 01 15:21:21.377434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:21.377434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:21.377434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:21.377434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:21.377434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:21.377434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:21.377434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:21.377434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:21.377434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:21.377434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:21.377434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:21.377434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:21.377434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:21.378646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:21.378646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 01 15:21:21.378646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 01 15:21:21.378646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:21.378646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:21.378646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 01 15:21:21.378646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 01 15:21:21.378646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a95321255a17439a9b37045ff247afc0. Aug 01 15:21:21.378646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:21.378646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 461/944] 10.222.0.83 () {60 vars in 1140 bytes} [Mon Aug 1 15:21:21 2022] POST /identity/v3/projects => generated 262 bytes in 268 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:21.378646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf88828-33bc-4198-b8b7-518f8556fac6 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:21.412243 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d900fd-a0c4-4a2d-bbc0-183b475b3539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:21.412974 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d900fd-a0c4-4a2d-bbc0-183b475b3539 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:21.413253 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d900fd-a0c4-4a2d-bbc0-183b475b3539 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:21.413603 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d900fd-a0c4-4a2d-bbc0-183b475b3539 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:21.414957 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d900fd-a0c4-4a2d-bbc0-183b475b3539 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:21.415261 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d900fd-a0c4-4a2d-bbc0-183b475b3539 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:21.416696 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d900fd-a0c4-4a2d-bbc0-183b475b3539 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:21.424110 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 484/945] 10.222.0.83 () {56 vars in 1122 bytes} [Mon Aug 1 15:21:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:21.434237 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d900fd-a0c4-4a2d-bbc0-183b475b3539 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:21.507027 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf88828-33bc-4198-b8b7-518f8556fac6 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:21.548671 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2fce96-9765-4e42-9b6e-70f8e47a2b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:21.549515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fce96-9765-4e42-9b6e-70f8e47a2b09 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:21.549905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fce96-9765-4e42-9b6e-70f8e47a2b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:21.550296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fce96-9765-4e42-9b6e-70f8e47a2b09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:21.559493 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2fce96-9765-4e42-9b6e-70f8e47a2b09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:21.561302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2fce96-9765-4e42-9b6e-70f8e47a2b09 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:21.574521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-cb2fce96-9765-4e42-9b6e-70f8e47a2b09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:21.574521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:21.574521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:21.574521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:21.574521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:21.574521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:21.574521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:21.574521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:21.574521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:21.574521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:21.574521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:21.574521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:21.574521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:21:21.574521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:21:21.574521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:21:21.576456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:21:21.576456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:21.576456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:21.576456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:21.576456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:21.576456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:21.576456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:21.576456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:21.576456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:21.576456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:21.576456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:21.576456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:21.576456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:21.576456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:21.576456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:21.576456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:21.576456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:21.578364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:21:21.578364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:21:21.578364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:21:21.578364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:21:21.578364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:21:21.578364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:21:21.578364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:21.578364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:21.582509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 462/946] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:21:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:21.583573 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2fce96-9765-4e42-9b6e-70f8e47a2b09 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:21.623803 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d900fd-a0c4-4a2d-bbc0-183b475b3539 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:21.635880 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:21.652800 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:21.671631 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887afd8-66ee-4188-b6c9-ae221e78e314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:21.672607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887afd8-66ee-4188-b6c9-ae221e78e314 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:21.673172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887afd8-66ee-4188-b6c9-ae221e78e314 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:21.673544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887afd8-66ee-4188-b6c9-ae221e78e314 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:21.675033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887afd8-66ee-4188-b6c9-ae221e78e314 None admin] RBAC: Adding query filter params (interface=public, service_id=c8a2f28ca6c34f03b90080cd11989a14, region_id=RegionOne) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:21.675453 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887afd8-66ee-4188-b6c9-ae221e78e314 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:21.677032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887afd8-66ee-4188-b6c9-ae221e78e314 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:21.688467 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 485/947] 10.222.0.83 () {56 vars in 1251 bytes} [Mon Aug 1 15:21:21 2022] GET /identity/v3/endpoints?service_id=c8a2f28ca6c34f03b90080cd11989a14&interface=public®ion_id=RegionOne => generated 194 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:21.689690 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887afd8-66ee-4188-b6c9-ae221e78e314 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:21.739967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2fce96-9765-4e42-9b6e-70f8e47a2b09 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:21.780785 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970fa4c-e164-4d36-ae3e-a02109dab74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:21.781548 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970fa4c-e164-4d36-ae3e-a02109dab74b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:21.781765 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970fa4c-e164-4d36-ae3e-a02109dab74b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:21.782072 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970fa4c-e164-4d36-ae3e-a02109dab74b None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:21.782898 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8970fa4c-e164-4d36-ae3e-a02109dab74b None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:21.783293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8970fa4c-e164-4d36-ae3e-a02109dab74b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:21.785029 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8970fa4c-e164-4d36-ae3e-a02109dab74b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:21.814080 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 463/948] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Aug 1 15:21:21 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:21.834387 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887afd8-66ee-4188-b6c9-ae221e78e314 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:21.878557 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5ed43-ab2b-4962-9165-d3f0bd01246e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:21.879535 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5ed43-ab2b-4962-9165-d3f0bd01246e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:21.880024 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5ed43-ab2b-4962-9165-d3f0bd01246e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:21.880504 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5ed43-ab2b-4962-9165-d3f0bd01246e None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:21.881473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b5ed43-ab2b-4962-9165-d3f0bd01246e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:21.883527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b5ed43-ab2b-4962-9165-d3f0bd01246e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:21.896098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 486/949] 10.222.0.83 () {56 vars in 1087 bytes} [Mon Aug 1 15:21:21 2022] GET /identity/v3/services => generated 1497 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:21:23.174004 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a52b50-519d-46a0-bdd2-c5b070316956 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:23.175062 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a52b50-519d-46a0-bdd2-c5b070316956 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:23.175549 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a52b50-519d-46a0-bdd2-c5b070316956 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:23.177566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 464/950] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:23.183661 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be20637-d93d-41db-b9ec-e7745e222b95 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:23.184643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be20637-d93d-41db-b9ec-e7745e222b95 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:23.185153 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be20637-d93d-41db-b9ec-e7745e222b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:23.202955 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef6c32-9c87-44e1-9679-f88ec9f4e309 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:23.203276 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef6c32-9c87-44e1-9679-f88ec9f4e309 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:23.203507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef6c32-9c87-44e1-9679-f88ec9f4e309 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:23.205173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 465/951] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:21:23.207848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2be20637-d93d-41db-b9ec-e7745e222b95 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:23.209484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2be20637-d93d-41db-b9ec-e7745e222b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:23.210538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a14a2-66e1-4036-87d8-44c45faa6ed5 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:23.210829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a14a2-66e1-4036-87d8-44c45faa6ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:23.211647 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a14a2-66e1-4036-87d8-44c45faa6ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:23.235122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4b2a14a2-66e1-4036-87d8-44c45faa6ed5 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:23.236774 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4b2a14a2-66e1-4036-87d8-44c45faa6ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:23.277597 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:23.318002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 487/952] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:21:23.318174 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 466/953] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:21:23.334525 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a14a2-66e1-4036-87d8-44c45faa6ed5 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:23.336921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be20637-d93d-41db-b9ec-e7745e222b95 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:23.359605 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:23.475088 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be20637-d93d-41db-b9ec-e7745e222b95 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:23.477057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a14a2-66e1-4036-87d8-44c45faa6ed5 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:23.522122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a1eed-e17f-4425-8520-0bc80bafb430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:23.523331 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a1eed-e17f-4425-8520-0bc80bafb430 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:23.523863 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a1eed-e17f-4425-8520-0bc80bafb430 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:23.524705 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a1eed-e17f-4425-8520-0bc80bafb430 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:23.536436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07392f70-7b2d-4242-8207-7848d390a942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:23.537370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07392f70-7b2d-4242-8207-7848d390a942 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:23.537821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07392f70-7b2d-4242-8207-7848d390a942 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:23.538276 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07392f70-7b2d-4242-8207-7848d390a942 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:23.539847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07392f70-7b2d-4242-8207-7848d390a942 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:23.540553 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a1eed-e17f-4425-8520-0bc80bafb430 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:21:23.542166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07392f70-7b2d-4242-8207-7848d390a942 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:23.547248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-ee5a1eed-e17f-4425-8520-0bc80bafb430 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:21:23.547248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:23.547248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:23.547248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:23.547248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:23.547248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:23.547248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:23.547248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:23.547248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:23.547248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:23.547248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:23.547248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:23.547248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:21:23.547248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:21:23.547248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:21:23.547248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:21:23.548264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:21:23.548264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:21:23.548264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:21:23.548264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:21:23.548264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:23.548264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:23.548264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:23.548264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:23.548264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:23.548264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:23.548264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:23.548264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:23.548264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:23.548264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:23.548264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:23.548264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:23.549171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:23.549171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:23.549171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:23.549171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:23.549171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:21:23.549171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:21:23.549171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:21:23.549171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:21:23.549171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:21:23.549171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:21:23.549171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:21:23.549171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:23.557114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 488/954] 10.222.0.83 () {56 vars in 1120 bytes} [Mon Aug 1 15:21:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:21:23.562996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a1eed-e17f-4425-8520-0bc80bafb430 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:23.563345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-07392f70-7b2d-4242-8207-7848d390a942 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:21:23.563345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:23.563345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:21:23.563345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:21:23.563345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:21:23.563345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:21:23.563345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:21:23.563345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:23.563345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:21:23.563345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:23.563345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:23.563345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:23.563345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:23.563345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:23.563345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:23.563345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:23.563345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:23.564595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:23.564595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:23.564595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:23.564595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:23.564595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:23.564595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:23.564595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:21:23.564595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:21:23.564595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:21:23.564595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:21:23.564595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:23.564595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:23.564595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:23.564595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:23.564595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:23.564595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:23.565577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:23.565577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:23.565577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:23.565577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:23.565577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:23.565577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:23.565577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:23.565577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:23.565577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:23.565577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:23.565577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:21:23.565577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:23.565577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:21:23.565577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:23.570017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 467/955] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:21:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:21:23.571719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07392f70-7b2d-4242-8207-7848d390a942 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:23.620151 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:23.675674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:23.697191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07392f70-7b2d-4242-8207-7848d390a942 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:23.702601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a1eed-e17f-4425-8520-0bc80bafb430 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:23.744062 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7860d3c-e763-43e2-9c2e-9ebfb045a5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:23.745200 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7860d3c-e763-43e2-9c2e-9ebfb045a5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:23.745701 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7860d3c-e763-43e2-9c2e-9ebfb045a5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:23.746192 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbab83-2d06-4cc1-a845-9324c042dede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:23.746333 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7860d3c-e763-43e2-9c2e-9ebfb045a5b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:23.746903 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbab83-2d06-4cc1-a845-9324c042dede None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:23.747254 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbab83-2d06-4cc1-a845-9324c042dede None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:23.747603 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbab83-2d06-4cc1-a845-9324c042dede None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:23.747904 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7860d3c-e763-43e2-9c2e-9ebfb045a5b9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:23.748401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7860d3c-e763-43e2-9c2e-9ebfb045a5b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:23.748504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbab83-2d06-4cc1-a845-9324c042dede None admin] RBAC: Adding query filter params (name=compute) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:23.748793 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbab83-2d06-4cc1-a845-9324c042dede None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:23.750292 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbab83-2d06-4cc1-a845-9324c042dede None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:23.750435 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7860d3c-e763-43e2-9c2e-9ebfb045a5b9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:23.761516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 468/956] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Aug 1 15:21:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:23.763082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbab83-2d06-4cc1-a845-9324c042dede None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:23.765972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 489/957] 10.222.0.83 () {56 vars in 1115 bytes} [Mon Aug 1 15:21:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:23.767824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7860d3c-e763-43e2-9c2e-9ebfb045a5b9 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:23.888173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:23.893190 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7860d3c-e763-43e2-9c2e-9ebfb045a5b9 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:23.905280 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbab83-2d06-4cc1-a845-9324c042dede None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:23.938329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefd3c0-6500-4abe-9010-ad3a13f83fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:23.939634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefd3c0-6500-4abe-9010-ad3a13f83fcd None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:23.940380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefd3c0-6500-4abe-9010-ad3a13f83fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:23.940906 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefd3c0-6500-4abe-9010-ad3a13f83fcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:23.954523 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c2741-6314-4ab1-91e6-eb78f02809f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:23.955932 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c2741-6314-4ab1-91e6-eb78f02809f6 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:23.956745 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c2741-6314-4ab1-91e6-eb78f02809f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:23.956905 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aefd3c0-6500-4abe-9010-ad3a13f83fcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:23.957495 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c2741-6314-4ab1-91e6-eb78f02809f6 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:23.958612 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c2741-6314-4ab1-91e6-eb78f02809f6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:23.959176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aefd3c0-6500-4abe-9010-ad3a13f83fcd None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:23.959493 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c2741-6314-4ab1-91e6-eb78f02809f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:23.962214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c2741-6314-4ab1-91e6-eb78f02809f6 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:23.977958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 469/958] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Aug 1 15:21:23 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:23.981643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-9aefd3c0-6500-4abe-9010-ad3a13f83fcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:23.981643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:23.981643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:23.981643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:23.981643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:23.981643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:23.981643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:23.981643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:23.981643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:23.981643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:23.981643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:23.981643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:23.981643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:23.981643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:23.981643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:23.982840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:23.982840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:23.982840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:23.982840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:23.982840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:23.982840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:23.982840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:23.982840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:23.982840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:23.982840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:23.982840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:23.982840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:23.982840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:23.982840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:23.982840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:23.982840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:23.982840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:23.984148 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:23.984148 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:23.984148 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:23.984148 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:23.989727 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 490/959] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:23.991113 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c2741-6314-4ab1-91e6-eb78f02809f6 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:24.022163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:24.139452 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c2741-6314-4ab1-91e6-eb78f02809f6 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:24.180726 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92a4d9-2f03-4e94-8945-f113f2ff4b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:24.181484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92a4d9-2f03-4e94-8945-f113f2ff4b64 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:24.181654 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92a4d9-2f03-4e94-8945-f113f2ff4b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:24.181950 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92a4d9-2f03-4e94-8945-f113f2ff4b64 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:24.182253 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7dc7e-7685-45ef-bf29-de46a9b71e72 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:24.182547 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7dc7e-7685-45ef-bf29-de46a9b71e72 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:24.182649 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92a4d9-2f03-4e94-8945-f113f2ff4b64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:24.182831 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7dc7e-7685-45ef-bf29-de46a9b71e72 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:24.183010 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92a4d9-2f03-4e94-8945-f113f2ff4b64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:24.184663 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92a4d9-2f03-4e94-8945-f113f2ff4b64 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:24.187143 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 491/960] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:24.191257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8c879-442a-46f1-bbfc-52e94ed15970 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:24.191805 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8c879-442a-46f1-bbfc-52e94ed15970 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:24.192375 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8c879-442a-46f1-bbfc-52e94ed15970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:24.199438 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 470/961] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:24.207388 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92a4d9-2f03-4e94-8945-f113f2ff4b64 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:24.222714 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-46b8c879-442a-46f1-bbfc-52e94ed15970 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:24.224424 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-46b8c879-442a-46f1-bbfc-52e94ed15970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:24.297418 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:24.324692 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 492/962] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:21:24.348863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92a4d9-2f03-4e94-8945-f113f2ff4b64 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:24.360062 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8c879-442a-46f1-bbfc-52e94ed15970 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:24.399823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7033d66-c65d-4d6d-9ed9-2f5b3330868d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:24.400876 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7033d66-c65d-4d6d-9ed9-2f5b3330868d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:24.401362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7033d66-c65d-4d6d-9ed9-2f5b3330868d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:24.401854 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7033d66-c65d-4d6d-9ed9-2f5b3330868d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:24.411210 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7033d66-c65d-4d6d-9ed9-2f5b3330868d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:24.413249 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7033d66-c65d-4d6d-9ed9-2f5b3330868d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:24.441415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-e7033d66-c65d-4d6d-9ed9-2f5b3330868d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:21:24.441415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:24.441415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:24.441415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:24.441415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:24.441415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:24.441415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:24.441415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:24.441415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:24.441415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:24.441415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:24.441415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:24.441415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:21:24.441415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:21:24.441415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:21:24.441415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:21:24.442741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:24.442741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:24.442741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:21:24.442741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:24.442741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:21:24.442741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:24.442741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:24.442741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:24.442741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:24.442741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:24.442741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:24.442741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:24.442741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:24.442741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:24.442741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:24.442741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:24.442741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:24.442741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:24.443724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:24.443724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:24.443724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:21:24.443724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:21:24.443724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:21:24.443724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:21:24.443724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:21:24.443724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:21:24.443724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:21:24.443724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:24.444290 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 471/963] 10.222.0.83 () {56 vars in 1117 bytes} [Mon Aug 1 15:21:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:24.451734 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7033d66-c65d-4d6d-9ed9-2f5b3330868d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:24.479981 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:24.505394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8c879-442a-46f1-bbfc-52e94ed15970 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:24.561612 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8f5e6-92ce-49fb-b8a3-0debc1281627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:24.562875 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8f5e6-92ce-49fb-b8a3-0debc1281627 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:24.563533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8f5e6-92ce-49fb-b8a3-0debc1281627 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:24.564013 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8f5e6-92ce-49fb-b8a3-0debc1281627 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:24.564957 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8f5e6-92ce-49fb-b8a3-0debc1281627 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:24.566735 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8f5e6-92ce-49fb-b8a3-0debc1281627 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:24.578511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-71b8f5e6-92ce-49fb-b8a3-0debc1281627 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:21:24.578511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:24.578511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:21:24.578511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:21:24.578511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:21:24.578511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:21:24.578511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:21:24.578511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:24.578511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:21:24.578511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:24.578511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:24.578511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:24.578511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:24.578511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:24.578511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:24.578511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:24.578511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:24.579662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:24.579662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:24.579662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:24.579662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:24.579662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:24.579662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:24.579662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:21:24.579662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:21:24.579662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:21:24.579662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:21:24.579662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:24.579662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:24.579662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:24.579662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:24.579662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:24.579662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:24.580555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:24.580555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:24.580555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:24.580555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:24.580555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:24.580555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:24.580555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:24.580555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:24.580555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:24.580555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:24.580555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:21:24.580555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:24.580555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:21:24.580555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:24.589325 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 493/964] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Aug 1 15:21:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:21:24.590700 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8f5e6-92ce-49fb-b8a3-0debc1281627 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:24.616503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7033d66-c65d-4d6d-9ed9-2f5b3330868d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:24.662675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:24.669951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cc3b8-d639-4e0e-9f80-9bf269dc2da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:24.670599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cc3b8-d639-4e0e-9f80-9bf269dc2da9 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:24.670884 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cc3b8-d639-4e0e-9f80-9bf269dc2da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:24.671207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cc3b8-d639-4e0e-9f80-9bf269dc2da9 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:24.672153 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0cc3b8-d639-4e0e-9f80-9bf269dc2da9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:24.672457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0cc3b8-d639-4e0e-9f80-9bf269dc2da9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:24.674213 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0cc3b8-d639-4e0e-9f80-9bf269dc2da9 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:24.703201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 472/965] 10.222.0.83 () {56 vars in 1149 bytes} [Mon Aug 1 15:21:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:24.704587 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cc3b8-d639-4e0e-9f80-9bf269dc2da9 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:24.738473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8f5e6-92ce-49fb-b8a3-0debc1281627 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:24.776275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40dd4e-3ba6-4c30-aa2f-8f3c33b6c7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:24.776903 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40dd4e-3ba6-4c30-aa2f-8f3c33b6c7fe None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:24.777180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40dd4e-3ba6-4c30-aa2f-8f3c33b6c7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:24.777450 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40dd4e-3ba6-4c30-aa2f-8f3c33b6c7fe None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:24.778156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40dd4e-3ba6-4c30-aa2f-8f3c33b6c7fe None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:24.778456 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40dd4e-3ba6-4c30-aa2f-8f3c33b6c7fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:24.779849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40dd4e-3ba6-4c30-aa2f-8f3c33b6c7fe None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:24.790086 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 494/966] 10.222.0.83 () {56 vars in 1122 bytes} [Mon Aug 1 15:21:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:24.791206 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40dd4e-3ba6-4c30-aa2f-8f3c33b6c7fe None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:24.805331 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:24.819275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cc3b8-d639-4e0e-9f80-9bf269dc2da9 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:24.856585 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785d571-547e-470b-aed0-518d7a44472b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:24.857218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785d571-547e-470b-aed0-518d7a44472b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:24.857526 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785d571-547e-470b-aed0-518d7a44472b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:24.857802 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785d571-547e-470b-aed0-518d7a44472b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:24.919629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40dd4e-3ba6-4c30-aa2f-8f3c33b6c7fe None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:24.926252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:24.940444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785d571-547e-470b-aed0-518d7a44472b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:24.941288 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785d571-547e-470b-aed0-518d7a44472b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:24.960733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237eae2-ae49-4bdd-9ee6-7b8686a5144f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:24.961665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237eae2-ae49-4bdd-9ee6-7b8686a5144f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:24.961976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237eae2-ae49-4bdd-9ee6-7b8686a5144f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:24.962251 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237eae2-ae49-4bdd-9ee6-7b8686a5144f None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:24.963583 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1237eae2-ae49-4bdd-9ee6-7b8686a5144f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:24.963583 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1237eae2-ae49-4bdd-9ee6-7b8686a5144f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:24.965192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1237eae2-ae49-4bdd-9ee6-7b8686a5144f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:24.970886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 495/967] 10.222.0.83 () {56 vars in 1122 bytes} [Mon Aug 1 15:21:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:24.972194 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaae19c-a431-450b-a6d0-02a122dfb45d None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:24.972546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaae19c-a431-450b-a6d0-02a122dfb45d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:24.972902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaae19c-a431-450b-a6d0-02a122dfb45d None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:24.974554 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 496/968] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:24.977641 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaae19c-a431-450b-a6d0-02a122dfb45d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:25.060025 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 473/969] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:21:24 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:21:25.061969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe69da-1b4d-491e-a2d9-422e6e30486e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:25.062543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe69da-1b4d-491e-a2d9-422e6e30486e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:25.062987 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe69da-1b4d-491e-a2d9-422e6e30486e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:25.075075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:25.084239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-0bfe69da-1b4d-491e-a2d9-422e6e30486e None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:25.085590 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0bfe69da-1b4d-491e-a2d9-422e6e30486e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:25.104294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaae19c-a431-450b-a6d0-02a122dfb45d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:25.145110 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca133b7-dbc9-4147-96b1-1bed0d48b34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:25.145741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca133b7-dbc9-4147-96b1-1bed0d48b34a None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:25.146294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca133b7-dbc9-4147-96b1-1bed0d48b34a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:25.146671 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca133b7-dbc9-4147-96b1-1bed0d48b34a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:25.147914 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca133b7-dbc9-4147-96b1-1bed0d48b34a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:25.149867 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca133b7-dbc9-4147-96b1-1bed0d48b34a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:25.166938 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 474/970] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:25.168284 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe69da-1b4d-491e-a2d9-422e6e30486e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:25.194512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 497/971] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:21:24 2022] POST /identity/v3/endpoints => generated 349 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:21:25.203177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:25.209648 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca133b7-dbc9-4147-96b1-1bed0d48b34a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:25.317359 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe69da-1b4d-491e-a2d9-422e6e30486e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:25.359817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca133b7-dbc9-4147-96b1-1bed0d48b34a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:25.366528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f6e18-ec03-4687-be42-37af79ffe013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:25.366961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f6e18-ec03-4687-be42-37af79ffe013 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:25.367931 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f6e18-ec03-4687-be42-37af79ffe013 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:25.369186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f6e18-ec03-4687-be42-37af79ffe013 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:25.384913 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f6e18-ec03-4687-be42-37af79ffe013 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:25.385735 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f6e18-ec03-4687-be42-37af79ffe013 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:25.396528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:25.409246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-ea0f6e18-ec03-4687-be42-37af79ffe013 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:25.409246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:25.409246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:25.409246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:25.409246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:25.409246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:25.409246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:25.409246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:25.409246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:25.409246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:25.409246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:25.409246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:25.409246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:25.409246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:25.409246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:25.410312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:25.410312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:25.410312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:25.410312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:25.410312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:25.410312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:25.410312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:25.410312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:25.410312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:25.410312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:25.410312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:25.410312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:25.410312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:25.410312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:25.410312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:25.410312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:25.410312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:25.411232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:25.411232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:25.411232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:25.411232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:25.411455 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364d3b4-523c-4515-91e1-d447567be854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:25.411546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 475/972] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:25.411635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364d3b4-523c-4515-91e1-d447567be854 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:25.411635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364d3b4-523c-4515-91e1-d447567be854 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:25.411635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364d3b4-523c-4515-91e1-d447567be854 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:25.412299 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364d3b4-523c-4515-91e1-d447567be854 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:25.413786 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364d3b4-523c-4515-91e1-d447567be854 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:25.416788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: WARNING keystone.server.flask.common [None req-3364d3b4-523c-4515-91e1-d447567be854 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 15:21:25.418852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f6e18-ec03-4687-be42-37af79ffe013 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:25.431315 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.sql.core [None req-3364d3b4-523c-4515-91e1-d447567be854 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 15:21:25.431315 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 15:21:25.431315 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [parameters: {'user_id': '367e3c3137b24ce88fd65d316236c3e1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 15:21:25.431315 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80348) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 15:21:25.439938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-3364d3b4-523c-4515-91e1-d447567be854 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 15:21:25.439938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:25.439938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 15:21:25.439938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 15:21:25.439938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 15:21:25.439938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 15:21:25.439938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 15:21:25.439938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 15:21:25.439938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 15:21:25.439938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application conn.query(q) Aug 01 15:21:25.439938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 15:21:25.439938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 15:21:25.439938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 15:21:25.439938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application result.read() Aug 01 15:21:25.439938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 15:21:25.440943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 15:21:25.440943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 15:21:25.440943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 15:21:25.440943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 15:21:25.440943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 15:21:25.440943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 15:21:25.440943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 15:21:25.440943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 15:21:25.440943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:25.440943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 15:21:25.440943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:25.440943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:25.440943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 15:21:25.440943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 15:21:25.440943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 01 15:21:25.440943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 01 15:21:25.440943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 15:21:25.440943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application next(self.gen) Aug 01 15:21:25.441991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 15:21:25.441991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application yield resource Aug 01 15:21:25.441991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 15:21:25.441991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application next(self.gen) Aug 01 15:21:25.441991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 15:21:25.441991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 15:21:25.441991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 15:21:25.441991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 15:21:25.441991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 15:21:25.441991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise self.value Aug 01 15:21:25.441991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 15:21:25.441991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 15:21:25.441991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 15:21:25.441991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application session.commit() Aug 01 15:21:25.441991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 15:21:25.441991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 15:21:25.441991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 15:21:25.442924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 15:21:25.442924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 15:21:25.442924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self.session.flush() Aug 01 15:21:25.442924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 15:21:25.442924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 15:21:25.442924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 15:21:25.442924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 15:21:25.442924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 15:21:25.442924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application compat.raise_( Aug 01 15:21:25.442924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 15:21:25.442924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception Aug 01 15:21:25.442924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 15:21:25.442924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 15:21:25.442924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 15:21:25.442924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 15:21:25.442924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 15:21:25.442924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 15:21:25.444808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 15:21:25.444808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 15:21:25.444808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 01 15:21:25.444808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 01 15:21:25.444808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 15:21:25.444808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 15:21:25.444808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 15:21:25.444808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 15:21:25.444808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 15:21:25.444808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 15:21:25.444808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 15:21:25.444808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 15:21:25.444808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 15:21:25.444808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 15:21:25.444808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 15:21:25.444808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception Aug 01 15:21:25.446878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 15:21:25.446878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 15:21:25.446878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 15:21:25.446878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 15:21:25.446878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 15:21:25.446878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 15:21:25.446878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 15:21:25.446878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application conn.query(q) Aug 01 15:21:25.446878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 15:21:25.446878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 15:21:25.446878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 15:21:25.446878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application result.read() Aug 01 15:21:25.446878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 15:21:25.446878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 15:21:25.446878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 15:21:25.446878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 15:21:25.448400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 15:21:25.448400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 15:21:25.448400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 15:21:25.448400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 15:21:25.448400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 15:21:25.448400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 15:21:25.448400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [parameters: {'user_id': '367e3c3137b24ce88fd65d316236c3e1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 15:21:25.448400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 15:21:25.448400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:25.448400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:25.448400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:25.448400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:25.448400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:25.448400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:25.448400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:25.450366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:25.450366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:25.450366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:25.450366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:25.450366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:25.450366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:25.450366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:25.450366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 01 15:21:25.450366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 01 15:21:25.450366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:25.450366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:25.450366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:21:25.450366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:25.450366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:21:25.450366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:25.450366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 01 15:21:25.450366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 01 15:21:25.452667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 01 15:21:25.452667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 01 15:21:25.452667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 15:21:25.452667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 15:21:25.452667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 15:21:25.452667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:25.452667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 498/973] 10.222.0.83 () {60 vars in 1131 bytes} [Mon Aug 1 15:21:25 2022] POST /identity/v3/users => generated 160 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:25.452667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364d3b4-523c-4515-91e1-d447567be854 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:25.516058 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:25.558061 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f6e18-ec03-4687-be42-37af79ffe013 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:25.582409 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364d3b4-523c-4515-91e1-d447567be854 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:25.605363 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553589fd-e219-4f26-8d47-407db46f9763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:25.606221 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553589fd-e219-4f26-8d47-407db46f9763 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:25.606480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553589fd-e219-4f26-8d47-407db46f9763 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:25.606764 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553589fd-e219-4f26-8d47-407db46f9763 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:25.607472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553589fd-e219-4f26-8d47-407db46f9763 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:25.607771 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553589fd-e219-4f26-8d47-407db46f9763 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:25.609318 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553589fd-e219-4f26-8d47-407db46f9763 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:25.630123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 476/974] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:25.631404 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553589fd-e219-4f26-8d47-407db46f9763 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:25.645651 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c05ae-41b6-4b7c-a38a-a704c01cbcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:25.646355 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c05ae-41b6-4b7c-a38a-a704c01cbcfd None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:25.646633 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c05ae-41b6-4b7c-a38a-a704c01cbcfd None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:25.646910 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c05ae-41b6-4b7c-a38a-a704c01cbcfd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:25.656291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c05ae-41b6-4b7c-a38a-a704c01cbcfd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:25.658257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c05ae-41b6-4b7c-a38a-a704c01cbcfd None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:25.671247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-de9c05ae-41b6-4b7c-a38a-a704c01cbcfd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:21:25.671247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:25.671247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:25.671247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:25.671247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:25.671247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:25.671247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:25.671247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:25.671247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:25.671247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:25.671247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:25.671247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:25.671247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:21:25.671247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:21:25.671247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:21:25.671247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:21:25.672284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:25.672284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:25.672284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:21:25.672284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:25.672284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:21:25.672284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:25.672284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:25.672284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:25.672284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:25.672284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:25.672284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:25.672284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:25.672284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:25.672284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:25.672284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:25.672284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:25.672284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:25.672284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:25.673468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:25.673468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:25.673468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:21:25.673468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:21:25.673468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:21:25.673468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:21:25.673468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:21:25.673468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:21:25.673468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:21:25.673468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:25.677699 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 499/975] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:21:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:25.679065 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c05ae-41b6-4b7c-a38a-a704c01cbcfd None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:25.745918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:25.782344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553589fd-e219-4f26-8d47-407db46f9763 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:25.820223 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:25.830392 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a9efc-714d-4ed6-a453-5793e59840ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:25.831424 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a9efc-714d-4ed6-a453-5793e59840ec None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:25.834328 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c05ae-41b6-4b7c-a38a-a704c01cbcfd None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:25.834905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a9efc-714d-4ed6-a453-5793e59840ec None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:25.835758 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a9efc-714d-4ed6-a453-5793e59840ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:25.845470 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a9efc-714d-4ed6-a453-5793e59840ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:25.847501 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a9efc-714d-4ed6-a453-5793e59840ec None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:25.880893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-a92a9efc-714d-4ed6-a453-5793e59840ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:25.880893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:25.880893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:25.880893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:25.880893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:25.880893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:25.880893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:25.880893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:25.880893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:25.880893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:25.880893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:25.880893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:25.880893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:21:25.880893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:21:25.880893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:21:25.882151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:21:25.882151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:25.882151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:25.882151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:25.882151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:25.882151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:25.882151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:25.882151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:25.882151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:25.882151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:25.882151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:25.882151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:25.882151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:25.882151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:25.882151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:25.882151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:25.882151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:25.883195 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:21:25.883195 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:21:25.883195 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:21:25.883195 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:21:25.883195 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:21:25.883195 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:21:25.883195 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:25.883195 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:25.884326 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc4c4c-3875-45fd-88bb-64e3e90fdb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:25.885189 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc4c4c-3875-45fd-88bb-64e3e90fdb74 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:25.885636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc4c4c-3875-45fd-88bb-64e3e90fdb74 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:25.886130 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc4c4c-3875-45fd-88bb-64e3e90fdb74 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:25.886481 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 477/976] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:21:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:21:25.887345 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc4c4c-3875-45fd-88bb-64e3e90fdb74 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:25.888255 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc4c4c-3875-45fd-88bb-64e3e90fdb74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:25.889203 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a9efc-714d-4ed6-a453-5793e59840ec None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:25.890393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc4c4c-3875-45fd-88bb-64e3e90fdb74 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:25.950407 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 500/977] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Aug 1 15:21:25 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:26.018721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a9efc-714d-4ed6-a453-5793e59840ec None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:26.063584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac161c3-bf6b-4ee9-8006-c244f125f35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:26.064649 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac161c3-bf6b-4ee9-8006-c244f125f35e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:26.065133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac161c3-bf6b-4ee9-8006-c244f125f35e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:26.065596 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac161c3-bf6b-4ee9-8006-c244f125f35e None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:26.066743 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac161c3-bf6b-4ee9-8006-c244f125f35e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:26.067252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac161c3-bf6b-4ee9-8006-c244f125f35e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:26.068990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac161c3-bf6b-4ee9-8006-c244f125f35e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:26.084779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 478/978] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:21:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:26.091826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc4c4c-3875-45fd-88bb-64e3e90fdb74 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:26.194340 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d19e36-7bbe-4ede-b387-d745d2b284be None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:26.194688 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d19e36-7bbe-4ede-b387-d745d2b284be None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:26.194908 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d19e36-7bbe-4ede-b387-d745d2b284be None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:26.196686 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 479/979] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:26.203626 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102ddc9-2d14-470e-8130-654919fd2776 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:26.204453 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102ddc9-2d14-470e-8130-654919fd2776 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:26.204453 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102ddc9-2d14-470e-8130-654919fd2776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:26.212236 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc4c4c-3875-45fd-88bb-64e3e90fdb74 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:26.222942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:26.235184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-5102ddc9-2d14-470e-8130-654919fd2776 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:26.237203 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5102ddc9-2d14-470e-8130-654919fd2776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:26.279377 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82d8c6-d932-4cf8-8ce5-09b35e2ebfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:26.280434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82d8c6-d932-4cf8-8ce5-09b35e2ebfa7 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:26.281182 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82d8c6-d932-4cf8-8ce5-09b35e2ebfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:26.282019 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82d8c6-d932-4cf8-8ce5-09b35e2ebfa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:26.282831 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82d8c6-d932-4cf8-8ce5-09b35e2ebfa7 None admin] RBAC: Adding query filter params (role.id=37a24cea29194c7ca77701607e008914, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=f36c6987ef774a1684c55f885e2a4f67) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:26.283143 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82d8c6-d932-4cf8-8ce5-09b35e2ebfa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:26.284615 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82d8c6-d932-4cf8-8ce5-09b35e2ebfa7 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:26.293802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 501/980] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:21:26 2022] GET /identity/v3/role_assignments?user.id=f36c6987ef774a1684c55f885e2a4f67&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=37a24cea29194c7ca77701607e008914 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:26.304453 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:26.332294 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 480/981] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:26.355986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82d8c6-d932-4cf8-8ce5-09b35e2ebfa7 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:26.481403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82d8c6-d932-4cf8-8ce5-09b35e2ebfa7 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:26.520226 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92faffa5-51ee-4709-848c-9e501913a9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:26.520865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92faffa5-51ee-4709-848c-9e501913a9b6 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:26.521147 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92faffa5-51ee-4709-848c-9e501913a9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:26.521420 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92faffa5-51ee-4709-848c-9e501913a9b6 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:26.522174 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92faffa5-51ee-4709-848c-9e501913a9b6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:26.524101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92faffa5-51ee-4709-848c-9e501913a9b6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:26.545285 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 502/982] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Aug 1 15:21:26 2022] POST /identity/v3/services => generated 242 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:21:27.406189 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44907c-f83a-4fd3-9184-cc3a10e6ff00 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:27.407045 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44907c-f83a-4fd3-9184-cc3a10e6ff00 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:27.407204 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44907c-f83a-4fd3-9184-cc3a10e6ff00 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:27.408979 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 481/983] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:27.415624 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d519f86-737d-45cb-8e2f-fa5ed9bd1b31 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:27.416294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d519f86-737d-45cb-8e2f-fa5ed9bd1b31 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:27.416294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d519f86-737d-45cb-8e2f-fa5ed9bd1b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:27.436543 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-3d519f86-737d-45cb-8e2f-fa5ed9bd1b31 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:27.438077 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3d519f86-737d-45cb-8e2f-fa5ed9bd1b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:27.466682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:27.534049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 503/984] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:27 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:27.553750 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44907c-f83a-4fd3-9184-cc3a10e6ff00 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:27.669214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44907c-f83a-4fd3-9184-cc3a10e6ff00 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:27.706313 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9568f7-165d-4883-a2b5-9f97869dc86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:27.706992 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9568f7-165d-4883-a2b5-9f97869dc86d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:27.707271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9568f7-165d-4883-a2b5-9f97869dc86d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:27.707541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9568f7-165d-4883-a2b5-9f97869dc86d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:27.708179 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9568f7-165d-4883-a2b5-9f97869dc86d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:27.709701 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9568f7-165d-4883-a2b5-9f97869dc86d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:27.720934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-4a9568f7-165d-4883-a2b5-9f97869dc86d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:21:27.720934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:27.720934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:21:27.720934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:21:27.720934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:21:27.720934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:21:27.720934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:21:27.720934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:27.720934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:21:27.720934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:27.720934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:27.720934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:27.720934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:27.720934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:27.720934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:27.720934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:27.720934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:27.722444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:27.722444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:27.722444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:27.722444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:27.722444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:27.722444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:27.722444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:21:27.722444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:21:27.722444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:21:27.722444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:21:27.722444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:27.722444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:27.722444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:27.722444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:27.722444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:27.722444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:27.723328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:27.723328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:27.723328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:27.723328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:27.723328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:27.723328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:27.723328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:27.723328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:27.723328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:27.723328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:27.723328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:21:27.723328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:27.723328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:21:27.723328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:27.724909 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 482/985] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Aug 1 15:21:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:21:27.729118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d519f86-737d-45cb-8e2f-fa5ed9bd1b31 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:27.835358 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d519f86-737d-45cb-8e2f-fa5ed9bd1b31 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:27.875013 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768ea4b-41a5-4735-8df1-cfaa57cba423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:27.875648 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768ea4b-41a5-4735-8df1-cfaa57cba423 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:27.875928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768ea4b-41a5-4735-8df1-cfaa57cba423 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:27.876200 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768ea4b-41a5-4735-8df1-cfaa57cba423 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:27.876880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768ea4b-41a5-4735-8df1-cfaa57cba423 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:27.877181 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768ea4b-41a5-4735-8df1-cfaa57cba423 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:27.878659 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768ea4b-41a5-4735-8df1-cfaa57cba423 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:27.889309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 504/986] 10.222.0.83 () {56 vars in 1122 bytes} [Mon Aug 1 15:21:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:27.890767 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9568f7-165d-4883-a2b5-9f97869dc86d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:27.943823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:28.002214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9568f7-165d-4883-a2b5-9f97869dc86d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:28.038798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627548a1-9a06-4578-9043-27b4e120ab98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:28.039458 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627548a1-9a06-4578-9043-27b4e120ab98 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:28.039742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627548a1-9a06-4578-9043-27b4e120ab98 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:28.040019 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627548a1-9a06-4578-9043-27b4e120ab98 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:28.040697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627548a1-9a06-4578-9043-27b4e120ab98 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:28.041000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627548a1-9a06-4578-9043-27b4e120ab98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:28.042595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627548a1-9a06-4578-9043-27b4e120ab98 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:28.048214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 483/987] 10.222.0.83 () {56 vars in 1122 bytes} [Mon Aug 1 15:21:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:28.055357 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768ea4b-41a5-4735-8df1-cfaa57cba423 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:28.082903 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:28.178558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768ea4b-41a5-4735-8df1-cfaa57cba423 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:28.220598 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5be05-8eda-4903-b803-348740c91061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:28.221122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5be05-8eda-4903-b803-348740c91061 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:28.221403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5be05-8eda-4903-b803-348740c91061 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:28.221679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5be05-8eda-4903-b803-348740c91061 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:28.222473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5be05-8eda-4903-b803-348740c91061 None admin] RBAC: Adding query filter params (interface=admin, service_id=c8a2f28ca6c34f03b90080cd11989a14, region_id=RegionOne) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:28.222778 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5be05-8eda-4903-b803-348740c91061 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:28.224380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5be05-8eda-4903-b803-348740c91061 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:28.229941 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 505/988] 10.222.0.83 () {56 vars in 1249 bytes} [Mon Aug 1 15:21:28 2022] GET /identity/v3/endpoints?service_id=c8a2f28ca6c34f03b90080cd11989a14&interface=admin®ion_id=RegionOne => generated 193 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:28.235513 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627548a1-9a06-4578-9043-27b4e120ab98 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:28.352657 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627548a1-9a06-4578-9043-27b4e120ab98 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:28.364692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:28.394419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f63752-94f5-4c97-91bc-ec11b1e0aad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:28.395107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f63752-94f5-4c97-91bc-ec11b1e0aad2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:28.395454 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f63752-94f5-4c97-91bc-ec11b1e0aad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:28.395736 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f63752-94f5-4c97-91bc-ec11b1e0aad2 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:28.396565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f63752-94f5-4c97-91bc-ec11b1e0aad2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:28.397972 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f63752-94f5-4c97-91bc-ec11b1e0aad2 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:28.406266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 484/989] 10.222.0.83 () {56 vars in 1087 bytes} [Mon Aug 1 15:21:28 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:21:28.664850 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb97ec-ab75-49d8-b6d2-3f7f7e9d8fd8 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:28.665595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb97ec-ab75-49d8-b6d2-3f7f7e9d8fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:28.665595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb97ec-ab75-49d8-b6d2-3f7f7e9d8fd8 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:28.666979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 506/990] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:28.672552 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077b02f-a267-43da-a440-8a3d97374d20 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:28.672844 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077b02f-a267-43da-a440-8a3d97374d20 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:28.673124 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077b02f-a267-43da-a440-8a3d97374d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:28.694098 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c077b02f-a267-43da-a440-8a3d97374d20 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:28.695395 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c077b02f-a267-43da-a440-8a3d97374d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:28.719587 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd8971-f08f-4298-9385-13ca4c632e36 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:28.719855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd8971-f08f-4298-9385-13ca4c632e36 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:28.720232 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd8971-f08f-4298-9385-13ca4c632e36 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:28.725997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 507/991] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:21:28.727988 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d145d-4ba7-42ca-85eb-6d1ab04f23f8 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:28.728277 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d145d-4ba7-42ca-85eb-6d1ab04f23f8 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:28.728552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d145d-4ba7-42ca-85eb-6d1ab04f23f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:28.747562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-091d145d-4ba7-42ca-85eb-6d1ab04f23f8 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:28.748768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-091d145d-4ba7-42ca-85eb-6d1ab04f23f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:28.774164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 485/992] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:28.797181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077b02f-a267-43da-a440-8a3d97374d20 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:28.821696 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 508/993] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:28.844019 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d145d-4ba7-42ca-85eb-6d1ab04f23f8 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:28.860951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:28.915897 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077b02f-a267-43da-a440-8a3d97374d20 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:28.918188 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:28.952044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c52fa9-9aab-48a9-9e06-9afced6e791c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:28.952810 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c52fa9-9aab-48a9-9e06-9afced6e791c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:28.953095 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c52fa9-9aab-48a9-9e06-9afced6e791c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:28.953369 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c52fa9-9aab-48a9-9e06-9afced6e791c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:28.954037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c52fa9-9aab-48a9-9e06-9afced6e791c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:28.955452 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c52fa9-9aab-48a9-9e06-9afced6e791c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:28.965265 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d145d-4ba7-42ca-85eb-6d1ab04f23f8 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:28.965456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-f1c52fa9-9aab-48a9-9e06-9afced6e791c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:21:28.965456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:28.965456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:21:28.965456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:21:28.965456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:21:28.965456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:21:28.965456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:21:28.965456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:28.965456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:21:28.965456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:28.965456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:28.965456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:28.965456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:28.965456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:28.965456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:28.965456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:28.965456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:28.966503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:28.966503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:28.966503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:28.966503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:28.966503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:28.966503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:28.966503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:21:28.966503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:21:28.966503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:21:28.966503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:21:28.966503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:28.966503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:28.966503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:28.966503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:28.966503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:28.966503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:28.967366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:28.967366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:28.967366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:28.967366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:28.967366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:28.967366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:28.967366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:28.967366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:28.967366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:28.967366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:28.967366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:21:28.967366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:28.967366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:21:28.967366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:28.967366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 486/994] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:21:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:21:28.972447 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c52fa9-9aab-48a9-9e06-9afced6e791c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:29.012295 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8f8e2-bb07-4876-9dd6-5a0444d02fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:29.012951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8f8e2-bb07-4876-9dd6-5a0444d02fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:29.013227 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8f8e2-bb07-4876-9dd6-5a0444d02fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:29.013495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8f8e2-bb07-4876-9dd6-5a0444d02fe6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:29.014972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea8f8e2-bb07-4876-9dd6-5a0444d02fe6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:29.015954 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea8f8e2-bb07-4876-9dd6-5a0444d02fe6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:29.025765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-0ea8f8e2-bb07-4876-9dd6-5a0444d02fe6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:21:29.025765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:29.025765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:21:29.025765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:21:29.025765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:21:29.025765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:21:29.025765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:21:29.025765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:29.025765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:21:29.025765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:29.025765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:29.025765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:29.025765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:29.025765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:29.025765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:29.025765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:29.025765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:29.027046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:29.027046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:29.027046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:29.027046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:29.027046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:29.027046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:29.027046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:21:29.027046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:21:29.027046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:21:29.027046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:21:29.027046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:29.027046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:29.027046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:29.027046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:29.027046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:29.027046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:29.027926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:29.027926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:29.027926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:29.027926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:29.027926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:29.027926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:29.027926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:29.027926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:29.027926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:29.027926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:29.027926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:21:29.027926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:29.027926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:21:29.027926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:29.027926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 509/995] 10.222.0.83 () {56 vars in 1105 bytes} [Mon Aug 1 15:21:28 2022] GET /identity/v3/services/image => generated 86 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:21:29.033810 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8f8e2-bb07-4876-9dd6-5a0444d02fe6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:29.104981 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c52fa9-9aab-48a9-9e06-9afced6e791c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:29.129817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:29.146423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658dc46b-b714-4961-b5c8-b4c04bf71a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:29.147061 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dc46b-b714-4961-b5c8-b4c04bf71a3d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:29.147374 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dc46b-b714-4961-b5c8-b4c04bf71a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:29.147614 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dc46b-b714-4961-b5c8-b4c04bf71a3d None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:29.148921 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658dc46b-b714-4961-b5c8-b4c04bf71a3d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:29.149128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658dc46b-b714-4961-b5c8-b4c04bf71a3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:29.150555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658dc46b-b714-4961-b5c8-b4c04bf71a3d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:29.160482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 487/996] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Aug 1 15:21:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:29.160717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8f8e2-bb07-4876-9dd6-5a0444d02fe6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:29.161603 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658dc46b-b714-4961-b5c8-b4c04bf71a3d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:29.197337 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:29.198285 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9420370-9572-4560-9211-5fbdb4fe80e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:29.198916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9420370-9572-4560-9211-5fbdb4fe80e1 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:29.199211 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9420370-9572-4560-9211-5fbdb4fe80e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:29.199457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9420370-9572-4560-9211-5fbdb4fe80e1 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:29.200143 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9420370-9572-4560-9211-5fbdb4fe80e1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:29.200441 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9420370-9572-4560-9211-5fbdb4fe80e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:29.201815 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9420370-9572-4560-9211-5fbdb4fe80e1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:29.214306 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 510/997] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:29.215562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9420370-9572-4560-9211-5fbdb4fe80e1 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:29.282997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658dc46b-b714-4961-b5c8-b4c04bf71a3d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:29.318999 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb091e-c1bb-45be-bb4a-334603460a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:29.319629 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb091e-c1bb-45be-bb4a-334603460a14 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:29.319913 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb091e-c1bb-45be-bb4a-334603460a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:29.320210 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb091e-c1bb-45be-bb4a-334603460a14 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:29.321217 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb091e-c1bb-45be-bb4a-334603460a14 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:29.321551 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb091e-c1bb-45be-bb4a-334603460a14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:29.323086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb091e-c1bb-45be-bb4a-334603460a14 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:29.328443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 488/998] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Aug 1 15:21:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:29.333396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9420370-9572-4560-9211-5fbdb4fe80e1 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:29.334316 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb091e-c1bb-45be-bb4a-334603460a14 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:29.372280 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92880b-a9a5-4f7a-9382-2afc892b91d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:29.372913 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92880b-a9a5-4f7a-9382-2afc892b91d0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:29.373194 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92880b-a9a5-4f7a-9382-2afc892b91d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:29.373466 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92880b-a9a5-4f7a-9382-2afc892b91d0 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:29.374183 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92880b-a9a5-4f7a-9382-2afc892b91d0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:29.374492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92880b-a9a5-4f7a-9382-2afc892b91d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:29.375872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92880b-a9a5-4f7a-9382-2afc892b91d0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:29.380673 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 511/999] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:29 2022] GET /identity/v3/services?type=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:29.468137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:29.476790 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb091e-c1bb-45be-bb4a-334603460a14 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:29.523000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20e64d-459b-4654-b4e4-55a52cc63f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:29.524237 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20e64d-459b-4654-b4e4-55a52cc63f95 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:29.524522 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20e64d-459b-4654-b4e4-55a52cc63f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:29.524855 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20e64d-459b-4654-b4e4-55a52cc63f95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:29.525631 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20e64d-459b-4654-b4e4-55a52cc63f95 None admin] RBAC: Adding query filter params (interface=public, service_id=ea6bb0e9a56e42398278a832115007fe, region_id=RegionOne) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:29.525983 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20e64d-459b-4654-b4e4-55a52cc63f95 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:29.527446 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20e64d-459b-4654-b4e4-55a52cc63f95 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:29.533819 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 489/1000] 10.222.0.83 () {56 vars in 1251 bytes} [Mon Aug 1 15:21:29 2022] GET /identity/v3/endpoints?service_id=ea6bb0e9a56e42398278a832115007fe&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:29.540048 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92880b-a9a5-4f7a-9382-2afc892b91d0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:29.549993 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:29.656934 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92880b-a9a5-4f7a-9382-2afc892b91d0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:29.692261 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85409b-80bf-4d55-b74c-45c59a5eb22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:29.692971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85409b-80bf-4d55-b74c-45c59a5eb22b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:29.693204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85409b-80bf-4d55-b74c-45c59a5eb22b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:29.693421 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85409b-80bf-4d55-b74c-45c59a5eb22b None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:29.694123 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85409b-80bf-4d55-b74c-45c59a5eb22b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:29.696259 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85409b-80bf-4d55-b74c-45c59a5eb22b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:29.703064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 512/1001] 10.222.0.83 () {56 vars in 1087 bytes} [Mon Aug 1 15:21:29 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:21:30.530718 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d43dc-d6ea-4ea3-befc-f67c17afc606 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:30.531316 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d43dc-d6ea-4ea3-befc-f67c17afc606 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:30.531316 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d43dc-d6ea-4ea3-befc-f67c17afc606 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:30.532970 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 490/1002] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:30.538461 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004307d-96b6-406a-b5c4-e79eb283e439 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:30.538743 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004307d-96b6-406a-b5c4-e79eb283e439 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:30.539539 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004307d-96b6-406a-b5c4-e79eb283e439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:30.558953 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d004307d-96b6-406a-b5c4-e79eb283e439 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:30.560185 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d004307d-96b6-406a-b5c4-e79eb283e439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:30.632791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:30.643208 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 513/1003] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:30 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:30.662289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d43dc-d6ea-4ea3-befc-f67c17afc606 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:30.778631 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d43dc-d6ea-4ea3-befc-f67c17afc606 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:30.814303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d6ba6-316d-488a-9b30-ebcef391b546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:30.814842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d6ba6-316d-488a-9b30-ebcef391b546 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:30.815121 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d6ba6-316d-488a-9b30-ebcef391b546 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:30.815427 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d6ba6-316d-488a-9b30-ebcef391b546 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:30.816065 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d6ba6-316d-488a-9b30-ebcef391b546 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:30.817542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d6ba6-316d-488a-9b30-ebcef391b546 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:30.827473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-dd0d6ba6-316d-488a-9b30-ebcef391b546 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:21:30.827473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:30.827473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:21:30.827473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:21:30.827473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:21:30.827473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:21:30.827473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:21:30.827473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:30.827473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:21:30.827473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:30.827473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:30.827473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:30.827473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:30.827473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:30.827473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:30.827473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:30.827473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:30.834138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:30.834138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:30.834138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:30.834138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:30.834138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:30.834138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:30.834138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:21:30.834138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:21:30.834138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:21:30.834138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:21:30.834138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:30.834138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:30.834138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:30.834138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:30.834138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:30.834138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:30.835035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:30.835035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:30.835035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:30.835035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:30.835035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:30.835035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:30.835035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:30.835035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:30.835035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:30.835035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:30.835035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:21:30.835035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:30.835035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:21:30.835035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:30.835035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 491/1004] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Aug 1 15:21:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:30.840406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004307d-96b6-406a-b5c4-e79eb283e439 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:30.959737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004307d-96b6-406a-b5c4-e79eb283e439 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:30.999871 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe737130-72a5-4a2a-b04d-5d19a843df4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:31.000507 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe737130-72a5-4a2a-b04d-5d19a843df4e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:31.000791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe737130-72a5-4a2a-b04d-5d19a843df4e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:31.001064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe737130-72a5-4a2a-b04d-5d19a843df4e None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:31.001747 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe737130-72a5-4a2a-b04d-5d19a843df4e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:31.002111 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe737130-72a5-4a2a-b04d-5d19a843df4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:31.003488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe737130-72a5-4a2a-b04d-5d19a843df4e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:31.009373 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 514/1005] 10.222.0.83 () {56 vars in 1122 bytes} [Mon Aug 1 15:21:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:31.016175 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d6ba6-316d-488a-9b30-ebcef391b546 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:31.055358 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:31.139469 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d6ba6-316d-488a-9b30-ebcef391b546 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:31.178939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e757d-2753-4b33-9492-bcc50be39f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:31.179598 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e757d-2753-4b33-9492-bcc50be39f57 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:31.179884 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e757d-2753-4b33-9492-bcc50be39f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:31.180163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e757d-2753-4b33-9492-bcc50be39f57 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:31.180854 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e757d-2753-4b33-9492-bcc50be39f57 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:31.181164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e757d-2753-4b33-9492-bcc50be39f57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:31.182656 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e757d-2753-4b33-9492-bcc50be39f57 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:31.192537 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 492/1006] 10.222.0.83 () {56 vars in 1122 bytes} [Mon Aug 1 15:21:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:31.194271 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe737130-72a5-4a2a-b04d-5d19a843df4e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:31.268095 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:31.321456 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe737130-72a5-4a2a-b04d-5d19a843df4e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:31.360096 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f17e6-8a9b-4234-b9fe-151b3d8017cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:31.360722 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f17e6-8a9b-4234-b9fe-151b3d8017cf None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:31.361065 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f17e6-8a9b-4234-b9fe-151b3d8017cf None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:31.361343 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f17e6-8a9b-4234-b9fe-151b3d8017cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:31.362111 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f17e6-8a9b-4234-b9fe-151b3d8017cf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:31.363559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f17e6-8a9b-4234-b9fe-151b3d8017cf None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:31.390011 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 515/1007] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:21:31 2022] POST /identity/v3/endpoints => generated 325 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:21:31.497908 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c27a88-dd9a-4cbd-b5c4-36abee58ca6f None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:31.498280 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c27a88-dd9a-4cbd-b5c4-36abee58ca6f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:31.498692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c27a88-dd9a-4cbd-b5c4-36abee58ca6f None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:31.500200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 493/1008] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:21:31.506091 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcb1e3-1448-4a88-8a2c-eaaee955e5c7 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:31.506429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcb1e3-1448-4a88-8a2c-eaaee955e5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:31.506648 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcb1e3-1448-4a88-8a2c-eaaee955e5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:31.526925 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-08dcb1e3-1448-4a88-8a2c-eaaee955e5c7 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:31.528218 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-08dcb1e3-1448-4a88-8a2c-eaaee955e5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:31.612244 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 516/1009] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:31 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:31.638748 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c27a88-dd9a-4cbd-b5c4-36abee58ca6f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:31.747821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:31.767148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c27a88-dd9a-4cbd-b5c4-36abee58ca6f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:31.803065 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650fe4b-e16d-4966-8022-6360167e1e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:31.803713 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650fe4b-e16d-4966-8022-6360167e1e67 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:31.803998 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650fe4b-e16d-4966-8022-6360167e1e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:31.804272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650fe4b-e16d-4966-8022-6360167e1e67 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:31.805010 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650fe4b-e16d-4966-8022-6360167e1e67 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:31.806457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650fe4b-e16d-4966-8022-6360167e1e67 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:31.829921 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 494/1010] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Aug 1 15:21:31 2022] POST /identity/v3/services => generated 242 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:21:31.901817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fcdeca-f53f-44a3-91dd-ce8c297ab849 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:31.902383 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fcdeca-f53f-44a3-91dd-ce8c297ab849 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:31.902635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fcdeca-f53f-44a3-91dd-ce8c297ab849 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:31.904667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 517/1011] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:31.910791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059f875-59ee-4716-863e-2ec894f6f983 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:31.911127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059f875-59ee-4716-863e-2ec894f6f983 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:31.911381 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059f875-59ee-4716-863e-2ec894f6f983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:31.932926 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-5059f875-59ee-4716-863e-2ec894f6f983 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:31.934180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5059f875-59ee-4716-863e-2ec894f6f983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:32.021978 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 495/1012] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:31 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:32.047143 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fcdeca-f53f-44a3-91dd-ce8c297ab849 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:32.112871 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:32.173225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fcdeca-f53f-44a3-91dd-ce8c297ab849 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:32.211802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872dda6b-abc5-45fb-bc53-b0b2b9b6e53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:32.212459 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872dda6b-abc5-45fb-bc53-b0b2b9b6e53d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:32.212742 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872dda6b-abc5-45fb-bc53-b0b2b9b6e53d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:32.213018 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872dda6b-abc5-45fb-bc53-b0b2b9b6e53d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:32.213737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872dda6b-abc5-45fb-bc53-b0b2b9b6e53d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:32.215093 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872dda6b-abc5-45fb-bc53-b0b2b9b6e53d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:32.228071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-872dda6b-abc5-45fb-bc53-b0b2b9b6e53d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:21:32.228071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:32.228071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:21:32.228071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:21:32.228071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:21:32.228071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:21:32.228071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:21:32.228071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:32.228071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:21:32.228071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:32.228071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:32.228071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:32.228071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:32.228071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:32.228071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:32.228071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:32.228071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:32.229180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:32.229180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:32.229180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:32.229180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:32.229180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:32.229180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:32.229180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:21:32.229180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:21:32.229180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:21:32.229180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:21:32.229180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:32.229180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:32.229180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:32.229180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:32.229180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:32.229180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:32.230713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:32.230713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:32.230713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:32.230713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:32.230713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:32.230713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:32.230713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:32.230713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:32.230713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:32.230713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:32.230713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:21:32.230713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:32.230713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:21:32.230713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:32.231732 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 518/1013] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:21:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:32.235723 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059f875-59ee-4716-863e-2ec894f6f983 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:32.317525 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:32.348119 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059f875-59ee-4716-863e-2ec894f6f983 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:32.384380 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2827cd-4536-4ed5-bf46-5a799f6aa670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:32.385030 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2827cd-4536-4ed5-bf46-5a799f6aa670 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:32.385312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2827cd-4536-4ed5-bf46-5a799f6aa670 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:32.385583 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2827cd-4536-4ed5-bf46-5a799f6aa670 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:32.386310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2827cd-4536-4ed5-bf46-5a799f6aa670 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:32.386610 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2827cd-4536-4ed5-bf46-5a799f6aa670 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:32.388057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2827cd-4536-4ed5-bf46-5a799f6aa670 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:32.397936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 496/1014] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Aug 1 15:21:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:32.398970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872dda6b-abc5-45fb-bc53-b0b2b9b6e53d None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:32.508380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872dda6b-abc5-45fb-bc53-b0b2b9b6e53d None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:32.525008 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:32.548629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c9dae-cfd1-4b51-8f92-4c6b13be430b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:32.549605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c9dae-cfd1-4b51-8f92-4c6b13be430b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:32.550143 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c9dae-cfd1-4b51-8f92-4c6b13be430b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:32.550417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c9dae-cfd1-4b51-8f92-4c6b13be430b None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:32.551097 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c9dae-cfd1-4b51-8f92-4c6b13be430b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:32.551398 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c9dae-cfd1-4b51-8f92-4c6b13be430b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:32.552760 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c9dae-cfd1-4b51-8f92-4c6b13be430b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:32.559157 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 519/1015] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Aug 1 15:21:32 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:32.564617 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2827cd-4536-4ed5-bf46-5a799f6aa670 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:32.676323 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2827cd-4536-4ed5-bf46-5a799f6aa670 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:32.712921 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b6500-dab9-474f-be58-f785a5c3d449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:32.713537 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b6500-dab9-474f-be58-f785a5c3d449 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:32.713817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b6500-dab9-474f-be58-f785a5c3d449 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:32.714115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b6500-dab9-474f-be58-f785a5c3d449 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:32.714826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b6500-dab9-474f-be58-f785a5c3d449 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:32.716208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b6500-dab9-474f-be58-f785a5c3d449 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:32.725174 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:32.745924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 497/1016] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:21:32 2022] POST /identity/v3/endpoints => generated 334 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:21:33.577460 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4d1c8-38fb-4c80-8f05-86e74c892387 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:33.578206 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4d1c8-38fb-4c80-8f05-86e74c892387 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:33.578206 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4d1c8-38fb-4c80-8f05-86e74c892387 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:33.579611 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 520/1017] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:33.584772 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fb3c8-b4ce-4c13-8db7-a76df52d74e8 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:33.585112 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fb3c8-b4ce-4c13-8db7-a76df52d74e8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:33.585414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fb3c8-b4ce-4c13-8db7-a76df52d74e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:33.607360 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-430fb3c8-b4ce-4c13-8db7-a76df52d74e8 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:33.608595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-430fb3c8-b4ce-4c13-8db7-a76df52d74e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:33.694135 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 498/1018] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:33.715795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4d1c8-38fb-4c80-8f05-86e74c892387 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:33.856857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4d1c8-38fb-4c80-8f05-86e74c892387 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:33.901205 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea3f5f-9849-4a1a-b175-fa1c3f1fe0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:33.901869 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea3f5f-9849-4a1a-b175-fa1c3f1fe0f3 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:33.902185 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea3f5f-9849-4a1a-b175-fa1c3f1fe0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:33.902455 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea3f5f-9849-4a1a-b175-fa1c3f1fe0f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:33.903662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea3f5f-9849-4a1a-b175-fa1c3f1fe0f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:33.905110 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea3f5f-9849-4a1a-b175-fa1c3f1fe0f3 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:33.906986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 521/1019] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:33.912670 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fb3c8-b4ce-4c13-8db7-a76df52d74e8 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:33.967962 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a0e4a-9c35-4da6-bc95-1081e8024a18 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:33.968325 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a0e4a-9c35-4da6-bc95-1081e8024a18 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:33.968610 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a0e4a-9c35-4da6-bc95-1081e8024a18 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:33.970340 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 522/1020] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:21:33.975838 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51938b68-bd5c-4bf3-8566-07f60fb104fb None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:33.976120 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51938b68-bd5c-4bf3-8566-07f60fb104fb None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:33.976911 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51938b68-bd5c-4bf3-8566-07f60fb104fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:33.995617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-51938b68-bd5c-4bf3-8566-07f60fb104fb None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:33.996810 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-51938b68-bd5c-4bf3-8566-07f60fb104fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:34.025335 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fb3c8-b4ce-4c13-8db7-a76df52d74e8 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:34.046394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:34.051398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:34.077426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd84bdb-961b-4943-be96-3f2e418f277a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:34.078102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd84bdb-961b-4943-be96-3f2e418f277a None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:34.078391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd84bdb-961b-4943-be96-3f2e418f277a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:34.078672 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd84bdb-961b-4943-be96-3f2e418f277a None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:34.079412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd84bdb-961b-4943-be96-3f2e418f277a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:34.080868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd84bdb-961b-4943-be96-3f2e418f277a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:34.082740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 523/1021] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:34.108385 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51938b68-bd5c-4bf3-8566-07f60fb104fb None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:34.113567 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 499/1022] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Aug 1 15:21:33 2022] POST /identity/v3/projects => generated 311 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:21:34.257188 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51938b68-bd5c-4bf3-8566-07f60fb104fb None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:34.297931 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e0e35-dc8c-4b1b-9da1-3d452203aafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:34.298588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e0e35-dc8c-4b1b-9da1-3d452203aafe None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:34.298878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e0e35-dc8c-4b1b-9da1-3d452203aafe None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:34.299158 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e0e35-dc8c-4b1b-9da1-3d452203aafe None admin] PATH_INFO: `/v3/services/image` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:34.299774 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e0e35-dc8c-4b1b-9da1-3d452203aafe None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:34.301141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e0e35-dc8c-4b1b-9da1-3d452203aafe None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:34.310955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-2f4e0e35-dc8c-4b1b-9da1-3d452203aafe None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:21:34.310955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:34.310955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:21:34.310955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:21:34.310955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:21:34.310955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:21:34.310955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:21:34.310955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:34.310955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:21:34.310955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:34.310955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:34.310955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:34.310955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:34.310955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:34.310955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:34.310955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:34.310955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:34.312492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:34.312492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:34.312492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:34.312492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:34.312492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:34.312492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:34.312492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:21:34.312492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:21:34.312492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:21:34.312492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:21:34.312492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:34.312492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:34.312492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:34.312492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:34.312492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:34.312492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:34.313768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:34.313768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:34.313768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:34.313768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:34.313768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:34.313768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:34.313768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:34.313768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:34.313768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:34.313768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:34.313768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:21:34.313768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:34.313768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:21:34.313768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:34.313768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 524/1023] 10.222.0.83 () {56 vars in 1105 bytes} [Mon Aug 1 15:21:34 2022] GET /identity/v3/services/image => generated 86 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:34.318707 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd84bdb-961b-4943-be96-3f2e418f277a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:34.451214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd84bdb-961b-4943-be96-3f2e418f277a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:34.490408 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7a1bb-e7de-4a84-acbf-270ef9d03600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:34.491492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7a1bb-e7de-4a84-acbf-270ef9d03600 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:34.492018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7a1bb-e7de-4a84-acbf-270ef9d03600 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:34.492532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7a1bb-e7de-4a84-acbf-270ef9d03600 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:34.493551 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c7a1bb-e7de-4a84-acbf-270ef9d03600 None admin] RBAC: Adding query filter params (name=image) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:34.494143 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c7a1bb-e7de-4a84-acbf-270ef9d03600 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:34.495934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c7a1bb-e7de-4a84-acbf-270ef9d03600 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:34.503008 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 500/1024] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:34.510013 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e0e35-dc8c-4b1b-9da1-3d452203aafe None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:34.578649 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:34.647403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e0e35-dc8c-4b1b-9da1-3d452203aafe None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:34.700733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c82e7-cd5d-454b-b698-6b35bfa7be0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:34.701519 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c82e7-cd5d-454b-b698-6b35bfa7be0b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:34.701744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c82e7-cd5d-454b-b698-6b35bfa7be0b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:34.702080 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c82e7-cd5d-454b-b698-6b35bfa7be0b None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:34.702872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c82e7-cd5d-454b-b698-6b35bfa7be0b None admin] RBAC: Adding query filter params (type=image) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:34.703187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c82e7-cd5d-454b-b698-6b35bfa7be0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:34.704882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c82e7-cd5d-454b-b698-6b35bfa7be0b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:34.711735 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 525/1025] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:34.718494 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7a1bb-e7de-4a84-acbf-270ef9d03600 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:34.770600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:34.856219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7a1bb-e7de-4a84-acbf-270ef9d03600 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:34.896663 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d50716-2214-40aa-ae4b-d82b56367206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:34.897291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d50716-2214-40aa-ae4b-d82b56367206 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:34.897596 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d50716-2214-40aa-ae4b-d82b56367206 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:34.897864 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d50716-2214-40aa-ae4b-d82b56367206 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:34.899425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d50716-2214-40aa-ae4b-d82b56367206 None admin] RBAC: Adding query filter params (interface=public, service_id=4764809425914281a7b4d6344409ed73, region_id=RegionOne) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:34.899868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d50716-2214-40aa-ae4b-d82b56367206 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:34.901297 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d50716-2214-40aa-ae4b-d82b56367206 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:34.907376 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 501/1026] 10.222.0.83 () {56 vars in 1251 bytes} [Mon Aug 1 15:21:34 2022] GET /identity/v3/endpoints?service_id=4764809425914281a7b4d6344409ed73&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:34.913183 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c82e7-cd5d-454b-b698-6b35bfa7be0b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:35.000652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46919dd-a2d1-4e26-b978-047fd2b9d79a None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:35.000881 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46919dd-a2d1-4e26-b978-047fd2b9d79a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:35.001139 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46919dd-a2d1-4e26-b978-047fd2b9d79a None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:35.002521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 502/1027] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:35.008504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d4775-b595-4547-ae1b-0479c45293dd None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:35.009302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d4775-b595-4547-ae1b-0479c45293dd None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:35.009799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d4775-b595-4547-ae1b-0479c45293dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:35.027582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c82e7-cd5d-454b-b698-6b35bfa7be0b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:35.030386 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:35.032078 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-508d4775-b595-4547-ae1b-0479c45293dd None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:35.033274 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-508d4775-b595-4547-ae1b-0479c45293dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:35.072171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab90d09-51a8-4e82-8eca-0f18077baf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:35.072791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab90d09-51a8-4e82-8eca-0f18077baf00 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:35.073073 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab90d09-51a8-4e82-8eca-0f18077baf00 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:35.073347 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab90d09-51a8-4e82-8eca-0f18077baf00 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:35.074040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab90d09-51a8-4e82-8eca-0f18077baf00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:35.075404 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab90d09-51a8-4e82-8eca-0f18077baf00 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:35.082653 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 526/1028] 10.222.0.83 () {56 vars in 1087 bytes} [Mon Aug 1 15:21:34 2022] GET /identity/v3/services => generated 1957 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:21:35.123914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 503/1029] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:21:35.149065 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab90d09-51a8-4e82-8eca-0f18077baf00 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:35.284553 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab90d09-51a8-4e82-8eca-0f18077baf00 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:35.322957 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae639548-1dd9-4fae-86ce-98523e10a534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:35.323621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae639548-1dd9-4fae-86ce-98523e10a534 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:35.323900 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae639548-1dd9-4fae-86ce-98523e10a534 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:35.324172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae639548-1dd9-4fae-86ce-98523e10a534 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:35.336594 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae639548-1dd9-4fae-86ce-98523e10a534 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:21:35.342118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-ae639548-1dd9-4fae-86ce-98523e10a534 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:21:35.342118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:35.342118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:35.342118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:35.342118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:35.342118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:35.342118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:35.342118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:35.342118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:35.342118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:35.342118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:35.342118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:35.342118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:21:35.342118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:21:35.342118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:21:35.342118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:21:35.343366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:21:35.343366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:21:35.343366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:21:35.343366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:21:35.343366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:35.343366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:35.343366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:35.343366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:35.343366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:35.343366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:35.343366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:35.343366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:35.343366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:35.343366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:35.343366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:35.343366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:35.344928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:35.344928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:35.344928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:35.344928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:35.344928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:21:35.344928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:21:35.344928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:21:35.344928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:21:35.344928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:21:35.344928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:21:35.344928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:21:35.344928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:35.344928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 527/1030] 10.222.0.83 () {56 vars in 1120 bytes} [Mon Aug 1 15:21:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:35.352059 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d4775-b595-4547-ae1b-0479c45293dd None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:35.384300 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:35.459570 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d4775-b595-4547-ae1b-0479c45293dd None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:35.497374 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de643d-b74d-4d84-8d72-9151e9b4799b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:35.498918 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de643d-b74d-4d84-8d72-9151e9b4799b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:35.499701 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de643d-b74d-4d84-8d72-9151e9b4799b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:35.500368 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de643d-b74d-4d84-8d72-9151e9b4799b None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:35.501348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de643d-b74d-4d84-8d72-9151e9b4799b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:35.502145 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de643d-b74d-4d84-8d72-9151e9b4799b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:35.504180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de643d-b74d-4d84-8d72-9151e9b4799b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:35.529982 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 504/1031] 10.222.0.83 () {56 vars in 1115 bytes} [Mon Aug 1 15:21:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:35.532309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae639548-1dd9-4fae-86ce-98523e10a534 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:35.588910 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:35.672880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae639548-1dd9-4fae-86ce-98523e10a534 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:35.708743 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837dfec-bc90-4e00-ac6d-860cf45311c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:35.709382 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837dfec-bc90-4e00-ac6d-860cf45311c5 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:35.709664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837dfec-bc90-4e00-ac6d-860cf45311c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:35.709967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837dfec-bc90-4e00-ac6d-860cf45311c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:35.720798 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837dfec-bc90-4e00-ac6d-860cf45311c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:35.722270 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837dfec-bc90-4e00-ac6d-860cf45311c5 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:35.738664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-9837dfec-bc90-4e00-ac6d-860cf45311c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:35.738664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:35.738664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:35.738664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:35.738664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:35.738664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:35.738664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:35.738664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:35.738664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:35.738664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:35.738664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:35.738664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:35.738664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:35.738664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:35.738664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:35.739907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:35.739907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:35.739907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:35.739907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:35.739907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:35.739907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:35.739907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:35.739907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:35.739907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:35.739907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:35.739907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:35.739907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:35.739907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:35.739907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:35.739907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:35.739907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:35.739907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:35.748465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:35.748465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:35.748465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:35.748465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:35.748465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 528/1032] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:35.752516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de643d-b74d-4d84-8d72-9151e9b4799b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:35.857414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:35.868304 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de643d-b74d-4d84-8d72-9151e9b4799b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:35.912408 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b8894-877b-4513-907d-5f7d20188248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:35.913116 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b8894-877b-4513-907d-5f7d20188248 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:35.913402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b8894-877b-4513-907d-5f7d20188248 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:35.913718 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b8894-877b-4513-907d-5f7d20188248 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:35.914617 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b8894-877b-4513-907d-5f7d20188248 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:35.914923 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b8894-877b-4513-907d-5f7d20188248 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:35.916342 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b8894-877b-4513-907d-5f7d20188248 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:35.949080 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 505/1033] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:35.951268 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837dfec-bc90-4e00-ac6d-860cf45311c5 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:36.051929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:36.077801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837dfec-bc90-4e00-ac6d-860cf45311c5 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:36.117878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc8d25-c26b-4755-9548-5963b0231bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:36.118561 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc8d25-c26b-4755-9548-5963b0231bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:36.118839 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc8d25-c26b-4755-9548-5963b0231bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:36.119174 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc8d25-c26b-4755-9548-5963b0231bb9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:36.127564 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc8d25-c26b-4755-9548-5963b0231bb9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:36.129000 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc8d25-c26b-4755-9548-5963b0231bb9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:36.142156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-adbc8d25-c26b-4755-9548-5963b0231bb9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:21:36.142156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:36.142156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:36.142156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:36.142156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:36.142156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:36.142156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:36.142156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:36.142156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:36.142156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:36.142156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:36.142156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:36.142156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:21:36.142156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:21:36.142156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:21:36.142156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:21:36.143209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:36.143209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:36.143209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:21:36.143209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:36.143209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:21:36.143209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:36.143209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:36.143209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:36.143209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:36.143209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:36.143209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:36.143209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:36.143209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:36.143209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:36.143209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:36.143209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:36.143209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:36.143209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:36.151830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:36.151830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:36.151830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:21:36.151830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:21:36.151830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:21:36.151830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:21:36.151830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:21:36.151830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:21:36.151830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:21:36.151830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:36.151830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 529/1034] 10.222.0.83 () {56 vars in 1093 bytes} [Mon Aug 1 15:21:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:36.152894 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b8894-877b-4513-907d-5f7d20188248 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:36.285938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8a55b-c24c-4768-9351-202e7f7d9402 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:36.286246 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8a55b-c24c-4768-9351-202e7f7d9402 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:36.286526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8a55b-c24c-4768-9351-202e7f7d9402 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:36.287890 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 530/1035] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:36.293776 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee50cf-8b14-4b5d-abaf-2420e6cd25c6 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:36.294146 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee50cf-8b14-4b5d-abaf-2420e6cd25c6 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:36.294488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee50cf-8b14-4b5d-abaf-2420e6cd25c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:36.299335 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b8894-877b-4513-907d-5f7d20188248 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:36.315000 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-7cee50cf-8b14-4b5d-abaf-2420e6cd25c6 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:36.316536 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7cee50cf-8b14-4b5d-abaf-2420e6cd25c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:36.337990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e198f0-7d35-45a4-b9d8-9ca7848f09ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:36.338919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e198f0-7d35-45a4-b9d8-9ca7848f09ae None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:36.338919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e198f0-7d35-45a4-b9d8-9ca7848f09ae None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:36.339216 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e198f0-7d35-45a4-b9d8-9ca7848f09ae None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:36.339962 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e198f0-7d35-45a4-b9d8-9ca7848f09ae None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:36.340274 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e198f0-7d35-45a4-b9d8-9ca7848f09ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:36.341718 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e198f0-7d35-45a4-b9d8-9ca7848f09ae None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:36.359952 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 506/1036] 10.222.0.83 () {56 vars in 1133 bytes} [Mon Aug 1 15:21:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:36.365436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e198f0-7d35-45a4-b9d8-9ca7848f09ae None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:36.379656 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:36.398824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 531/1037] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:36.423982 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee50cf-8b14-4b5d-abaf-2420e6cd25c6 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:36.498288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:36.499927 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e198f0-7d35-45a4-b9d8-9ca7848f09ae None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:36.531743 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee50cf-8b14-4b5d-abaf-2420e6cd25c6 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:36.547797 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b7982-2c28-46b6-a706-75f28b7e322c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:36.548886 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b7982-2c28-46b6-a706-75f28b7e322c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:36.549353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b7982-2c28-46b6-a706-75f28b7e322c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:36.550088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b7982-2c28-46b6-a706-75f28b7e322c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:36.571278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90a089-166d-4a4f-a687-af9fad12c27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:36.572121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90a089-166d-4a4f-a687-af9fad12c27f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:36.572408 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90a089-166d-4a4f-a687-af9fad12c27f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:36.572729 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90a089-166d-4a4f-a687-af9fad12c27f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:36.574307 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90a089-166d-4a4f-a687-af9fad12c27f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:36.576055 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90a089-166d-4a4f-a687-af9fad12c27f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:36.578368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 532/1038] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:36.585209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90a089-166d-4a4f-a687-af9fad12c27f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:36.640391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b7982-2c28-46b6-a706-75f28b7e322c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:36.641943 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b7982-2c28-46b6-a706-75f28b7e322c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:36.655099 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:36.703806 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90a089-166d-4a4f-a687-af9fad12c27f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:36.756407 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8eb88-1358-4e7c-9bfe-8dd7a5857cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:36.757652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8eb88-1358-4e7c-9bfe-8dd7a5857cbd None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:36.758322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8eb88-1358-4e7c-9bfe-8dd7a5857cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:36.758914 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8eb88-1358-4e7c-9bfe-8dd7a5857cbd None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:36.759505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 507/1039] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:21:36 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 15:21:36.760868 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8eb88-1358-4e7c-9bfe-8dd7a5857cbd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:36.762831 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8eb88-1358-4e7c-9bfe-8dd7a5857cbd None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:36.766054 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b7982-2c28-46b6-a706-75f28b7e322c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:36.777380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:36.792343 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 533/1040] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Aug 1 15:21:36 2022] POST /identity/v3/users => generated 285 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:21:36.888871 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b7982-2c28-46b6-a706-75f28b7e322c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:36.903737 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:36.932355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99469d82-cd83-4df1-8085-908f4c96fe79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:36.933405 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99469d82-cd83-4df1-8085-908f4c96fe79 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:36.933925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99469d82-cd83-4df1-8085-908f4c96fe79 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:36.934409 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99469d82-cd83-4df1-8085-908f4c96fe79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:36.946627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99469d82-cd83-4df1-8085-908f4c96fe79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:36.948264 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99469d82-cd83-4df1-8085-908f4c96fe79 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:36.965751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-99469d82-cd83-4df1-8085-908f4c96fe79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:36.965751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:36.965751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:36.965751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:36.965751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:36.965751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:36.965751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:36.965751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:36.965751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:36.965751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:36.965751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:36.965751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:36.965751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:36.965751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:36.965751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:36.966919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:36.966919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:36.966919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:36.966919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:36.966919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:36.966919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:36.966919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:36.966919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:36.966919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:36.966919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:36.966919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:36.966919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:36.966919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:36.966919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:36.966919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:36.966919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:36.966919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:36.968039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:36.968039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:36.968039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:36.968039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:36.968039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 508/1041] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:36.974979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8eb88-1358-4e7c-9bfe-8dd7a5857cbd None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:37.093336 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8eb88-1358-4e7c-9bfe-8dd7a5857cbd None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:37.134831 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77995d38-16e6-40d7-85c7-1d96a8f829f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:37.135818 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77995d38-16e6-40d7-85c7-1d96a8f829f2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:37.136280 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77995d38-16e6-40d7-85c7-1d96a8f829f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:37.136743 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77995d38-16e6-40d7-85c7-1d96a8f829f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:37.137626 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77995d38-16e6-40d7-85c7-1d96a8f829f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:37.138158 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77995d38-16e6-40d7-85c7-1d96a8f829f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:37.139746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77995d38-16e6-40d7-85c7-1d96a8f829f2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:37.155043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 534/1042] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:37.160364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99469d82-cd83-4df1-8085-908f4c96fe79 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:37.201236 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfbe46-d7e9-4db5-ac52-75e7311a0e53 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:37.201574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfbe46-d7e9-4db5-ac52-75e7311a0e53 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:37.201791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfbe46-d7e9-4db5-ac52-75e7311a0e53 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:37.203337 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 535/1043] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:21:37.209511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d9a35-4e54-4cfd-8396-64c61fb3d8c7 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:37.209943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d9a35-4e54-4cfd-8396-64c61fb3d8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:37.210121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d9a35-4e54-4cfd-8396-64c61fb3d8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:37.232370 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-fa0d9a35-4e54-4cfd-8396-64c61fb3d8c7 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:37.233852 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fa0d9a35-4e54-4cfd-8396-64c61fb3d8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:37.285177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99469d82-cd83-4df1-8085-908f4c96fe79 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:37.293612 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:37.314264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 536/1044] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:21:37.323643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fde39c-dcd8-4a18-b03f-994db760ab77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:37.324294 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fde39c-dcd8-4a18-b03f-994db760ab77 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:37.324619 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fde39c-dcd8-4a18-b03f-994db760ab77 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:37.324859 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fde39c-dcd8-4a18-b03f-994db760ab77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:37.331902 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fde39c-dcd8-4a18-b03f-994db760ab77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:37.333367 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fde39c-dcd8-4a18-b03f-994db760ab77 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:37.337833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d9a35-4e54-4cfd-8396-64c61fb3d8c7 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:37.346114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-57fde39c-dcd8-4a18-b03f-994db760ab77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:37.346114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:37.346114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:37.346114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:37.346114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:37.346114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:37.346114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:37.346114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:37.346114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:37.346114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:37.346114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:37.346114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:37.346114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:21:37.346114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:21:37.346114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:21:37.347107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:21:37.347107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:37.347107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:37.347107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:37.347107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:37.347107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:37.347107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:37.347107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:37.347107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:37.347107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:37.347107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:37.347107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:37.347107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:37.347107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:37.347107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:37.347107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:37.347107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:37.348796 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:21:37.348796 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:21:37.348796 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:21:37.348796 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:21:37.348796 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:21:37.348796 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:21:37.348796 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:37.348796 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:37.348796 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 509/1045] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:21:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:37.355462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fde39c-dcd8-4a18-b03f-994db760ab77 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:37.395294 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:37.453776 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d9a35-4e54-4cfd-8396-64c61fb3d8c7 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:37.496078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d196d1c-f316-495a-985e-6045dc04ba0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:37.496832 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d196d1c-f316-495a-985e-6045dc04ba0e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:37.497159 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d196d1c-f316-495a-985e-6045dc04ba0e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:37.497425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d196d1c-f316-495a-985e-6045dc04ba0e None admin] PATH_INFO: `/v3/services/image` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:37.498190 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d196d1c-f316-495a-985e-6045dc04ba0e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:37.499672 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d196d1c-f316-495a-985e-6045dc04ba0e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:37.502045 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fde39c-dcd8-4a18-b03f-994db760ab77 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:37.510429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-8d196d1c-f316-495a-985e-6045dc04ba0e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:21:37.510429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:37.510429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:21:37.510429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:21:37.510429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:21:37.510429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:21:37.510429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:21:37.510429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:37.510429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:21:37.510429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:37.510429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:21:37.510429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:37.510429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:37.510429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:37.510429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:37.510429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:37.510429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:37.511828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:37.511828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:37.511828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:37.511828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:37.511828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:37.511828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:37.511828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:21:37.511828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:21:37.511828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:21:37.511828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:21:37.511828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:37.511828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:37.511828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:37.511828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:37.511828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:37.511828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:37.512802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:37.512802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:37.512802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:37.512802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:37.512802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:37.512802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:37.512802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:37.512802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:37.512802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:37.512802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:37.512802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:21:37.512802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:21:37.512802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:21:37.512802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:37.512802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 537/1046] 10.222.0.83 () {56 vars in 1105 bytes} [Mon Aug 1 15:21:37 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:21:37.519046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d196d1c-f316-495a-985e-6045dc04ba0e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:37.550487 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6fb27-ac7a-41cd-ae14-b002191049c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:37.551199 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6fb27-ac7a-41cd-ae14-b002191049c2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:37.551395 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6fb27-ac7a-41cd-ae14-b002191049c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:37.551647 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6fb27-ac7a-41cd-ae14-b002191049c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:37.552399 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6fb27-ac7a-41cd-ae14-b002191049c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:37.552709 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6fb27-ac7a-41cd-ae14-b002191049c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:37.554187 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6fb27-ac7a-41cd-ae14-b002191049c2 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:37.574857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 510/1047] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:21:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:37.580125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:37.584337 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6fb27-ac7a-41cd-ae14-b002191049c2 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:37.640721 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d196d1c-f316-495a-985e-6045dc04ba0e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:37.681013 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8771d7-beee-481e-8012-88d96bb4b382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:37.681935 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8771d7-beee-481e-8012-88d96bb4b382 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:37.682359 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8771d7-beee-481e-8012-88d96bb4b382 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:37.682762 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8771d7-beee-481e-8012-88d96bb4b382 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:37.684507 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8771d7-beee-481e-8012-88d96bb4b382 None admin] RBAC: Adding query filter params (name=image) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:37.684507 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8771d7-beee-481e-8012-88d96bb4b382 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:37.686399 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8771d7-beee-481e-8012-88d96bb4b382 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:37.699993 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 538/1048] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:37.701180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8771d7-beee-481e-8012-88d96bb4b382 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:37.701705 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:37.721321 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6fb27-ac7a-41cd-ae14-b002191049c2 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:37.760410 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d489eb-1501-4e59-9ef8-c4998952f05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:37.761150 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d489eb-1501-4e59-9ef8-c4998952f05d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:37.761391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d489eb-1501-4e59-9ef8-c4998952f05d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:37.761575 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d489eb-1501-4e59-9ef8-c4998952f05d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:37.762366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d489eb-1501-4e59-9ef8-c4998952f05d None admin] RBAC: Adding query filter params (role.id=37a24cea29194c7ca77701607e008914, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=25dc41ffc41c47fdbeec06110d04610c) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:37.762676 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d489eb-1501-4e59-9ef8-c4998952f05d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:37.764240 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d489eb-1501-4e59-9ef8-c4998952f05d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:37.770284 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 511/1049] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:21:37 2022] GET /identity/v3/role_assignments?user.id=25dc41ffc41c47fdbeec06110d04610c&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=37a24cea29194c7ca77701607e008914 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:37.818348 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8771d7-beee-481e-8012-88d96bb4b382 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:37.834541 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:37.860741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab28de-63f1-44f0-bd89-0acdbe077391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:37.861431 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab28de-63f1-44f0-bd89-0acdbe077391 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:37.861715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab28de-63f1-44f0-bd89-0acdbe077391 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:37.862016 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab28de-63f1-44f0-bd89-0acdbe077391 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:37.862708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab28de-63f1-44f0-bd89-0acdbe077391 None admin] RBAC: Adding query filter params (type=image) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:37.863011 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab28de-63f1-44f0-bd89-0acdbe077391 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:37.864417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab28de-63f1-44f0-bd89-0acdbe077391 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:37.875661 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 539/1050] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:37.877133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d489eb-1501-4e59-9ef8-c4998952f05d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:38.007191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d489eb-1501-4e59-9ef8-c4998952f05d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:38.070784 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf96b0-05a6-4aa8-bd3e-46039aee4026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:38.071683 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf96b0-05a6-4aa8-bd3e-46039aee4026 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:38.071967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf96b0-05a6-4aa8-bd3e-46039aee4026 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:38.072244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf96b0-05a6-4aa8-bd3e-46039aee4026 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:38.072975 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf96b0-05a6-4aa8-bd3e-46039aee4026 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:38.074541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf96b0-05a6-4aa8-bd3e-46039aee4026 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:38.097438 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 512/1051] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:21:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:21:38.989547 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16cfd5-5e3d-4901-aa96-4d3ac545f8d2 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:38.990100 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16cfd5-5e3d-4901-aa96-4d3ac545f8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:38.990178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16cfd5-5e3d-4901-aa96-4d3ac545f8d2 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:38.991523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 540/1052] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:38.996749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8ee03-4f7a-4619-9e95-819ea05f744d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:38.997153 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8ee03-4f7a-4619-9e95-819ea05f744d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:38.997342 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8ee03-4f7a-4619-9e95-819ea05f744d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:39.003650 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:39.017186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e3e8ee03-4f7a-4619-9e95-819ea05f744d None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:39.018423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e3e8ee03-4f7a-4619-9e95-819ea05f744d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:39.102070 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 513/1053] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:39.123784 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16cfd5-5e3d-4901-aa96-4d3ac545f8d2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:39.234881 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16cfd5-5e3d-4901-aa96-4d3ac545f8d2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:39.271648 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a61996-fdff-44b6-9082-83cc0a757d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:39.272229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a61996-fdff-44b6-9082-83cc0a757d03 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:39.272509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a61996-fdff-44b6-9082-83cc0a757d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:39.272781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a61996-fdff-44b6-9082-83cc0a757d03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:39.285475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a61996-fdff-44b6-9082-83cc0a757d03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:21:39.291101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-03a61996-fdff-44b6-9082-83cc0a757d03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:21:39.291101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:39.291101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:39.291101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:39.291101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:39.291101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:39.291101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:39.291101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:39.291101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:39.291101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:39.291101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:39.291101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:39.291101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:21:39.291101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:21:39.291101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:21:39.291101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:21:39.292030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:21:39.292030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:21:39.292030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:21:39.292030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:21:39.292030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:39.292030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:39.292030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:39.292030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:39.292030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:39.292030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:39.292030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:39.292030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:39.292030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:39.292030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:39.292030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:39.292030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:39.292883 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:39.292883 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:39.292883 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:39.292883 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:39.292883 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:21:39.292883 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:21:39.292883 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:21:39.292883 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:21:39.292883 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:21:39.292883 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:21:39.292883 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:21:39.292883 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:39.300609 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 541/1054] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:21:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:39.300766 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8ee03-4f7a-4619-9e95-819ea05f744d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:39.413525 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8ee03-4f7a-4619-9e95-819ea05f744d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:39.441616 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:39.452416 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f461a7-dd35-45d0-99f3-069de322fb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:39.453064 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f461a7-dd35-45d0-99f3-069de322fb69 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:39.453434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f461a7-dd35-45d0-99f3-069de322fb69 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:39.453714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f461a7-dd35-45d0-99f3-069de322fb69 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:39.454471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f461a7-dd35-45d0-99f3-069de322fb69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:39.454785 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f461a7-dd35-45d0-99f3-069de322fb69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:39.456166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f461a7-dd35-45d0-99f3-069de322fb69 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:39.465699 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 514/1055] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:21:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:39.471276 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a61996-fdff-44b6-9082-83cc0a757d03 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:39.511609 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:39.590868 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a61996-fdff-44b6-9082-83cc0a757d03 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:39.626262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e0fab-621a-47f2-89f3-8ddbd06d0b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:39.626973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e0fab-621a-47f2-89f3-8ddbd06d0b87 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:39.627255 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e0fab-621a-47f2-89f3-8ddbd06d0b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:39.627586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e0fab-621a-47f2-89f3-8ddbd06d0b87 None admin] PATH_INFO: `/v3/users/473ffcd293ba4fbba34e7e697268c28b` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:39.643096 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e0fab-621a-47f2-89f3-8ddbd06d0b87 None admin] RBAC: Authorizing `identity:get_user(user_id=473ffcd293ba4fbba34e7e697268c28b)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:39.645210 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e0fab-621a-47f2-89f3-8ddbd06d0b87 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:39.647806 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 542/1056] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:21:39 2022] GET /identity/v3/users/473ffcd293ba4fbba34e7e697268c28b => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:39.655725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f461a7-dd35-45d0-99f3-069de322fb69 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:39.765911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f461a7-dd35-45d0-99f3-069de322fb69 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:39.798134 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ca1a1-16e5-484c-a5a9-8d569a0f5f35 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:39.798420 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ca1a1-16e5-484c-a5a9-8d569a0f5f35 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:39.798700 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ca1a1-16e5-484c-a5a9-8d569a0f5f35 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:39.800225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 543/1057] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:21:39.802875 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d3288-80a4-4b46-84c7-0f0b2d81d13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:39.803508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d3288-80a4-4b46-84c7-0f0b2d81d13e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:39.804022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d3288-80a4-4b46-84c7-0f0b2d81d13e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:39.804022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d3288-80a4-4b46-84c7-0f0b2d81d13e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:39.805684 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbd9bd-b5c6-4d01-9b7b-1ad00730cacd None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:39.806006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbd9bd-b5c6-4d01-9b7b-1ad00730cacd None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:39.806295 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbd9bd-b5c6-4d01-9b7b-1ad00730cacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:39.831495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f4bbd9bd-b5c6-4d01-9b7b-1ad00730cacd None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:39.833335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f4bbd9bd-b5c6-4d01-9b7b-1ad00730cacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:39.869910 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:39.881115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d3288-80a4-4b46-84c7-0f0b2d81d13e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:39.883249 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d3288-80a4-4b46-84c7-0f0b2d81d13e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:39.937687 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 544/1058] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:21:39.962949 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbd9bd-b5c6-4d01-9b7b-1ad00730cacd None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:39.998234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:40.024240 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 515/1059] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:21:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:21:40.032788 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d3288-80a4-4b46-84c7-0f0b2d81d13e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:40.119412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbd9bd-b5c6-4d01-9b7b-1ad00730cacd None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:40.154102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:40.160791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fa29e-9db3-42bf-86d1-80120bf82ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:40.161717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fa29e-9db3-42bf-86d1-80120bf82ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:40.162030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fa29e-9db3-42bf-86d1-80120bf82ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:40.162309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fa29e-9db3-42bf-86d1-80120bf82ef8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:40.166586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d3288-80a4-4b46-84c7-0f0b2d81d13e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:40.175210 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4fa29e-9db3-42bf-86d1-80120bf82ef8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:21:40.181334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-3c4fa29e-9db3-42bf-86d1-80120bf82ef8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:21:40.181334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:40.181334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:40.181334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:40.181334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:40.181334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:40.181334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:40.181334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:40.181334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:40.181334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:40.181334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:40.181334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:40.181334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:21:40.181334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:21:40.181334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:21:40.181334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:21:40.182477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:21:40.182477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:21:40.182477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:21:40.182477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:21:40.182477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:40.182477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:40.182477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:40.182477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:40.182477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:40.182477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:40.182477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:40.182477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:40.182477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:40.182477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:40.182477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:40.182477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:40.186856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:40.186856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:40.186856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:40.186856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:40.186856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:21:40.186856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:21:40.186856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:21:40.186856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:21:40.186856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:21:40.186856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:21:40.186856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:21:40.186856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:40.186856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 545/1060] 10.222.0.83 () {56 vars in 1120 bytes} [Mon Aug 1 15:21:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:21:40.188613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fa29e-9db3-42bf-86d1-80120bf82ef8 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:40.216946 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f3bff-ccf0-45a3-a678-069f8e6e9a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:40.217603 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f3bff-ccf0-45a3-a678-069f8e6e9a4f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:40.217905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f3bff-ccf0-45a3-a678-069f8e6e9a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:40.218192 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f3bff-ccf0-45a3-a678-069f8e6e9a4f None admin] PATH_INFO: `/v3/projects/226905b77a6b41a9af28e4c148199a6b` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:40.219371 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f3bff-ccf0-45a3-a678-069f8e6e9a4f None admin] RBAC: Authorizing `identity:get_project(project_id=226905b77a6b41a9af28e4c148199a6b)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:40.220820 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f3bff-ccf0-45a3-a678-069f8e6e9a4f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:40.223731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 516/1061] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:21:40 2022] GET /identity/v3/projects/226905b77a6b41a9af28e4c148199a6b => generated 311 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:40.223878 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d00ffc-225c-430b-b977-6ee00e792885 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:40.224164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d00ffc-225c-430b-b977-6ee00e792885 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:40.224450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d00ffc-225c-430b-b977-6ee00e792885 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:40.225908 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 517/1062] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:40.228942 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d00ffc-225c-430b-b977-6ee00e792885 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:40.286210 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:40.306467 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fa29e-9db3-42bf-86d1-80120bf82ef8 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:40.340163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d00ffc-225c-430b-b977-6ee00e792885 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:40.342669 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35bdec4-b5b6-410a-b751-31356298ade2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:40.343296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35bdec4-b5b6-410a-b751-31356298ade2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:40.343575 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35bdec4-b5b6-410a-b751-31356298ade2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:40.343845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35bdec4-b5b6-410a-b751-31356298ade2 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:40.344551 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35bdec4-b5b6-410a-b751-31356298ade2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:40.344902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35bdec4-b5b6-410a-b751-31356298ade2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:40.346247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35bdec4-b5b6-410a-b751-31356298ade2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:40.354736 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 546/1063] 10.222.0.83 () {56 vars in 1115 bytes} [Mon Aug 1 15:21:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:40.355930 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f9d09-1bd6-4c5b-9321-8892f6659913 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:40.356216 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f9d09-1bd6-4c5b-9321-8892f6659913 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:40.356495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f9d09-1bd6-4c5b-9321-8892f6659913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:40.375101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-256f9d09-1bd6-4c5b-9321-8892f6659913 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:40.376286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-256f9d09-1bd6-4c5b-9321-8892f6659913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:40.376393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed98b27-0350-4880-92f9-bf7bc25b7ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:40.376917 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed98b27-0350-4880-92f9-bf7bc25b7ecf None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:40.377201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed98b27-0350-4880-92f9-bf7bc25b7ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:40.377478 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed98b27-0350-4880-92f9-bf7bc25b7ecf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:40.378478 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed98b27-0350-4880-92f9-bf7bc25b7ecf None admin] RBAC: Adding query filter params (role.id=7bdfa43dc1d046f7965ffe52b74bc58f, scope.project.id=226905b77a6b41a9af28e4c148199a6b, user.id=473ffcd293ba4fbba34e7e697268c28b) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:40.378794 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed98b27-0350-4880-92f9-bf7bc25b7ecf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:40.380991 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed98b27-0350-4880-92f9-bf7bc25b7ecf None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:40.387438 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 518/1064] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:21:40 2022] GET /identity/v3/role_assignments?user.id=473ffcd293ba4fbba34e7e697268c28b&scope.project.id=226905b77a6b41a9af28e4c148199a6b&role.id=7bdfa43dc1d046f7965ffe52b74bc58f => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:40.388322 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed98b27-0350-4880-92f9-bf7bc25b7ecf None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:40.428968 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:40.458356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 547/1065] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:40.491076 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f9d09-1bd6-4c5b-9321-8892f6659913 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:40.526567 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed98b27-0350-4880-92f9-bf7bc25b7ecf None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:40.571219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367d19b-3935-4843-b469-67010acd4722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:40.571880 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367d19b-3935-4843-b469-67010acd4722 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:40.572167 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367d19b-3935-4843-b469-67010acd4722 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:40.572439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367d19b-3935-4843-b469-67010acd4722 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:40.584612 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2367d19b-3935-4843-b469-67010acd4722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:40.592952 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2367d19b-3935-4843-b469-67010acd4722 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:40.594622 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:40.610826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-2367d19b-3935-4843-b469-67010acd4722 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:40.610826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:40.610826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:40.610826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:40.610826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:40.610826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:40.610826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:40.610826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:40.610826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:40.610826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:40.610826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:40.610826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:40.610826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:40.610826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:40.610826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:40.611914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:40.611914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:40.611914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:40.611914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:40.611914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:40.611914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:40.611914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:40.611914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:40.611914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:40.611914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:40.611914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:40.611914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:40.611914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:40.611914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:40.611914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:40.611914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:40.611914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:40.612924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:40.612924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:40.612924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:40.612924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:40.614742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 519/1066] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:40.614952 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f9d09-1bd6-4c5b-9321-8892f6659913 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:40.621457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367d19b-3935-4843-b469-67010acd4722 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:40.656083 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e3b48-5770-4d0c-9168-f12ecf3573a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:40.657014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e3b48-5770-4d0c-9168-f12ecf3573a6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:40.657450 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e3b48-5770-4d0c-9168-f12ecf3573a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:40.657724 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e3b48-5770-4d0c-9168-f12ecf3573a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:40.711706 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:40.753446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e3b48-5770-4d0c-9168-f12ecf3573a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:40.754972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e3b48-5770-4d0c-9168-f12ecf3573a6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:40.760722 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367d19b-3935-4843-b469-67010acd4722 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:40.812391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c23ee1-41f3-4c74-b107-40d16c25d998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:40.813761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c23ee1-41f3-4c74-b107-40d16c25d998 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:40.814208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c23ee1-41f3-4c74-b107-40d16c25d998 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:40.815002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c23ee1-41f3-4c74-b107-40d16c25d998 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:40.816205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c23ee1-41f3-4c74-b107-40d16c25d998 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:40.817018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c23ee1-41f3-4c74-b107-40d16c25d998 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:40.820108 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c23ee1-41f3-4c74-b107-40d16c25d998 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:40.836214 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:40.841842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 520/1067] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:40.849033 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c23ee1-41f3-4c74-b107-40d16c25d998 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:40.869717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 548/1068] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:21:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:21:40.880127 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e3b48-5770-4d0c-9168-f12ecf3573a6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:40.998841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c23ee1-41f3-4c74-b107-40d16c25d998 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:41.019011 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e3b48-5770-4d0c-9168-f12ecf3573a6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:41.044290 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816d464-4946-407e-8126-eeccbc0cf848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:41.044952 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816d464-4946-407e-8126-eeccbc0cf848 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:41.045248 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816d464-4946-407e-8126-eeccbc0cf848 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:41.045526 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816d464-4946-407e-8126-eeccbc0cf848 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:41.048171 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:41.055518 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816d464-4946-407e-8126-eeccbc0cf848 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:41.057163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816d464-4946-407e-8126-eeccbc0cf848 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:41.070484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-d816d464-4946-407e-8126-eeccbc0cf848 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:21:41.070484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:41.070484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:41.070484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:41.070484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:41.070484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:41.070484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:41.070484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:41.070484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:41.070484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:41.070484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:41.070484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:41.070484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:21:41.070484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:21:41.070484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:21:41.070484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:21:41.072391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:41.072391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:41.072391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:21:41.072391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:41.072391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:21:41.072391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:41.072391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:41.072391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:41.072391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:41.072391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:41.072391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:41.072391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:41.072391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:41.072391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:41.072391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:41.072391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:41.072391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:41.072391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:41.073925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:41.073925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:41.073925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:21:41.073925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:21:41.073925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:21:41.073925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:21:41.073925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:21:41.073925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:21:41.073925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:21:41.073925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:41.073925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 521/1069] 10.222.0.83 () {56 vars in 1093 bytes} [Mon Aug 1 15:21:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:41.078141 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816d464-4946-407e-8126-eeccbc0cf848 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:41.081181 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd6647-6890-4b44-94fa-8092c900d8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:41.082062 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd6647-6890-4b44-94fa-8092c900d8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:41.082462 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd6647-6890-4b44-94fa-8092c900d8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:41.082868 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd6647-6890-4b44-94fa-8092c900d8b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:41.098129 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd6647-6890-4b44-94fa-8092c900d8b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:41.100289 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd6647-6890-4b44-94fa-8092c900d8b4 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:41.119902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-c6cd6647-6890-4b44-94fa-8092c900d8b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:41.119902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:41.119902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:41.119902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:41.119902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:41.119902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:41.119902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:41.119902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:41.119902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:41.119902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:41.119902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:41.119902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:41.119902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:41.119902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:41.119902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:41.121057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:41.121057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:41.121057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:41.121057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:41.121057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:41.121057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:41.121057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:41.121057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:41.121057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:41.121057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:41.121057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:41.121057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:41.121057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:41.121057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:41.121057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:41.121057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:41.121057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:41.122079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:41.122079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:41.122079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:41.122079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:41.122079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 549/1070] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:41.126974 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd6647-6890-4b44-94fa-8092c900d8b4 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:41.160737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:41.203460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816d464-4946-407e-8126-eeccbc0cf848 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:41.241805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640573c-5b8a-437f-b980-83c10759184a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:41.242478 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640573c-5b8a-437f-b980-83c10759184a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:41.242767 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640573c-5b8a-437f-b980-83c10759184a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:41.243043 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640573c-5b8a-437f-b980-83c10759184a None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:41.243853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640573c-5b8a-437f-b980-83c10759184a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:41.244165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640573c-5b8a-437f-b980-83c10759184a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:41.245625 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640573c-5b8a-437f-b980-83c10759184a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:41.252239 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd6647-6890-4b44-94fa-8092c900d8b4 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:41.264968 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 522/1071] 10.222.0.83 () {56 vars in 1133 bytes} [Mon Aug 1 15:21:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:41.270741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640573c-5b8a-437f-b980-83c10759184a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:41.289418 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83fd358-1066-4352-87d4-2f7735c0a649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:41.291095 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83fd358-1066-4352-87d4-2f7735c0a649 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:41.291619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83fd358-1066-4352-87d4-2f7735c0a649 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:41.292201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83fd358-1066-4352-87d4-2f7735c0a649 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:41.293270 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83fd358-1066-4352-87d4-2f7735c0a649 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:41.293840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83fd358-1066-4352-87d4-2f7735c0a649 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:41.295511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83fd358-1066-4352-87d4-2f7735c0a649 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:41.310596 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 550/1072] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:41.339355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:41.399351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640573c-5b8a-437f-b980-83c10759184a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:41.439310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55329e8-2340-408f-a9ca-b5eae62e798f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:41.439984 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55329e8-2340-408f-a9ca-b5eae62e798f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:41.440363 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55329e8-2340-408f-a9ca-b5eae62e798f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:41.440517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55329e8-2340-408f-a9ca-b5eae62e798f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:41.461628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55329e8-2340-408f-a9ca-b5eae62e798f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:41.463502 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55329e8-2340-408f-a9ca-b5eae62e798f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:41.482554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-a55329e8-2340-408f-a9ca-b5eae62e798f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:41.482554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:41.482554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:41.482554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:41.482554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:41.482554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:41.482554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:41.482554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:41.482554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:41.482554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:41.482554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:41.482554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:41.482554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:41.482554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:41.482554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:41.484235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:41.484235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:41.484235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:41.484235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:41.484235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:41.484235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:41.484235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:41.484235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:41.484235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:41.484235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:41.484235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:41.484235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:41.484235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:41.484235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:41.484235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:41.484235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:41.484235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:41.485989 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:41.485989 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:41.485989 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:41.485989 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:41.485989 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 523/1073] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:21:41.490075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83fd358-1066-4352-87d4-2f7735c0a649 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:41.597169 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:41.611476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83fd358-1066-4352-87d4-2f7735c0a649 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:41.646842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca0f70-cbd5-4929-906a-bc4d3dea84f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:41.647460 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca0f70-cbd5-4929-906a-bc4d3dea84f7 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:41.647732 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca0f70-cbd5-4929-906a-bc4d3dea84f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:41.648001 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca0f70-cbd5-4929-906a-bc4d3dea84f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:41.649352 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca0f70-cbd5-4929-906a-bc4d3dea84f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:41.649648 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca0f70-cbd5-4929-906a-bc4d3dea84f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:41.651047 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca0f70-cbd5-4929-906a-bc4d3dea84f7 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:41.662466 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 551/1074] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:41.668362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55329e8-2340-408f-a9ca-b5eae62e798f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:41.772731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55329e8-2340-408f-a9ca-b5eae62e798f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:41.775791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:41.808438 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb53961-fc3b-4761-add9-6825e721cfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:41.809085 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb53961-fc3b-4761-add9-6825e721cfd7 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:41.809371 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb53961-fc3b-4761-add9-6825e721cfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:41.809647 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb53961-fc3b-4761-add9-6825e721cfd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:41.816473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb53961-fc3b-4761-add9-6825e721cfd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:41.817979 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb53961-fc3b-4761-add9-6825e721cfd7 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:41.830934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-6cb53961-fc3b-4761-add9-6825e721cfd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:41.830934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:41.830934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:41.830934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:41.830934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:41.830934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:41.830934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:41.830934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:41.830934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:41.830934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:41.830934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:41.830934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:41.830934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:21:41.830934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:21:41.830934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:21:41.832012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:21:41.832012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:41.832012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:41.832012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:41.832012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:41.832012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:41.832012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:41.832012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:41.832012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:41.832012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:41.832012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:41.832012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:41.832012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:41.832012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:41.832012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:41.832012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:41.832012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:41.833147 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:21:41.833147 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:21:41.833147 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:21:41.833147 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:21:41.833147 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:21:41.833147 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:21:41.833147 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:41.833147 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:41.834041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 524/1075] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:21:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:41.837915 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca0f70-cbd5-4929-906a-bc4d3dea84f7 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:41.940641 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca0f70-cbd5-4929-906a-bc4d3dea84f7 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:41.977272 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c2277-d1cd-44e8-ad44-cfa3b9b799d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:41.977925 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c2277-d1cd-44e8-ad44-cfa3b9b799d2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:41.978206 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c2277-d1cd-44e8-ad44-cfa3b9b799d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:41.978471 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c2277-d1cd-44e8-ad44-cfa3b9b799d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:41.979213 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c2277-d1cd-44e8-ad44-cfa3b9b799d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:41.979513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c2277-d1cd-44e8-ad44-cfa3b9b799d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:41.980973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c2277-d1cd-44e8-ad44-cfa3b9b799d2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:41.994600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 552/1076] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:21:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:42.000669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb53961-fc3b-4761-add9-6825e721cfd7 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:42.112736 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb53961-fc3b-4761-add9-6825e721cfd7 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:42.150980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18be01-4446-4a8f-935e-1c36032b6eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:42.151661 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18be01-4446-4a8f-935e-1c36032b6eaf None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:42.151949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18be01-4446-4a8f-935e-1c36032b6eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:42.152228 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18be01-4446-4a8f-935e-1c36032b6eaf None admin] PATH_INFO: `/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/25dc41ffc41c47fdbeec06110d04610c/roles/37a24cea29194c7ca77701607e008914` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:42.154581 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18be01-4446-4a8f-935e-1c36032b6eaf None admin] RBAC: Authorizing `identity:create_grant(project_id=1f4e27cc50bd4f9f8ced4cc544ab3064, user_id=25dc41ffc41c47fdbeec06110d04610c, role_id=37a24cea29194c7ca77701607e008914)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:42.156905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18be01-4446-4a8f-935e-1c36032b6eaf None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:42.170426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 525/1077] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:21:41 2022] PUT /identity/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/25dc41ffc41c47fdbeec06110d04610c/roles/37a24cea29194c7ca77701607e008914 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:21:42.452185 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe0f85-4ca9-4559-a9de-72d3de9181ba None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:42.452833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe0f85-4ca9-4559-a9de-72d3de9181ba None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:42.452833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe0f85-4ca9-4559-a9de-72d3de9181ba None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:42.454122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 553/1078] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:42.459284 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabecd98-321e-4677-8cd0-483f023a3a2f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:42.459648 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabecd98-321e-4677-8cd0-483f023a3a2f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:42.459937 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabecd98-321e-4677-8cd0-483f023a3a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:42.478669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-eabecd98-321e-4677-8cd0-483f023a3a2f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:42.479870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-eabecd98-321e-4677-8cd0-483f023a3a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:42.498806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:42.549867 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 526/1079] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:42.572920 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe0f85-4ca9-4559-a9de-72d3de9181ba None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:42.584043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:42.678610 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe0f85-4ca9-4559-a9de-72d3de9181ba None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:42.713805 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276d7cb-f44d-4394-8260-8d64022ffeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:42.714395 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276d7cb-f44d-4394-8260-8d64022ffeff None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:42.714676 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276d7cb-f44d-4394-8260-8d64022ffeff None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:42.714946 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276d7cb-f44d-4394-8260-8d64022ffeff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:42.725134 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276d7cb-f44d-4394-8260-8d64022ffeff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:21:42.730797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-c276d7cb-f44d-4394-8260-8d64022ffeff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:21:42.730797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:42.730797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:42.730797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:42.730797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:42.730797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:42.730797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:42.730797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:42.730797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:42.730797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:42.730797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:42.730797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:42.730797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:21:42.730797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:21:42.730797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:21:42.730797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:21:42.732057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:21:42.732057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:21:42.732057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:21:42.732057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:21:42.732057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:42.732057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:42.732057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:42.732057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:42.732057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:42.732057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:42.732057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:42.732057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:42.732057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:42.732057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:42.732057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:42.732057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:42.732998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:42.732998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:42.732998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:42.732998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:42.732998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:21:42.732998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:21:42.732998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:21:42.732998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:21:42.732998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:21:42.732998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:21:42.732998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:21:42.732998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:42.732998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 554/1080] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:21:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:42.738010 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabecd98-321e-4677-8cd0-483f023a3a2f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:42.837245 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabecd98-321e-4677-8cd0-483f023a3a2f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:42.874660 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd54b8-2287-4f06-ac2d-19e8d4b31e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:42.875415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd54b8-2287-4f06-ac2d-19e8d4b31e3a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:42.875804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd54b8-2287-4f06-ac2d-19e8d4b31e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:42.876173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd54b8-2287-4f06-ac2d-19e8d4b31e3a None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:42.877021 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd54b8-2287-4f06-ac2d-19e8d4b31e3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:42.877450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd54b8-2287-4f06-ac2d-19e8d4b31e3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:42.879099 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd54b8-2287-4f06-ac2d-19e8d4b31e3a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:42.888169 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 527/1081] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:21:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:42.895522 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276d7cb-f44d-4394-8260-8d64022ffeff None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:42.966229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:43.010242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276d7cb-f44d-4394-8260-8d64022ffeff None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:43.047395 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9cb12-dcf0-44a8-8ac4-d8b040f84f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:43.048173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9cb12-dcf0-44a8-8ac4-d8b040f84f59 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:43.048454 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9cb12-dcf0-44a8-8ac4-d8b040f84f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:43.048732 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9cb12-dcf0-44a8-8ac4-d8b040f84f59 None admin] PATH_INFO: `/v3/users/473ffcd293ba4fbba34e7e697268c28b` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:43.050164 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9cb12-dcf0-44a8-8ac4-d8b040f84f59 None admin] RBAC: Authorizing `identity:get_user(user_id=473ffcd293ba4fbba34e7e697268c28b)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:43.051482 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9cb12-dcf0-44a8-8ac4-d8b040f84f59 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:43.053322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 555/1082] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:21:42 2022] GET /identity/v3/users/473ffcd293ba4fbba34e7e697268c28b => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:43.059087 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd54b8-2287-4f06-ac2d-19e8d4b31e3a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:43.084281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:43.180816 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd54b8-2287-4f06-ac2d-19e8d4b31e3a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:43.220138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c46a8-0e22-4db3-98b4-624782753909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:43.220717 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c46a8-0e22-4db3-98b4-624782753909 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:43.221000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c46a8-0e22-4db3-98b4-624782753909 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:43.221281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c46a8-0e22-4db3-98b4-624782753909 None admin] PATH_INFO: `/v3/projects/226905b77a6b41a9af28e4c148199a6b` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:43.222492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c46a8-0e22-4db3-98b4-624782753909 None admin] RBAC: Authorizing `identity:get_project(project_id=226905b77a6b41a9af28e4c148199a6b)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:43.223943 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c46a8-0e22-4db3-98b4-624782753909 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:43.225642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 528/1083] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:21:43 2022] GET /identity/v3/projects/226905b77a6b41a9af28e4c148199a6b => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:43.232357 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9cb12-dcf0-44a8-8ac4-d8b040f84f59 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:43.343938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9cb12-dcf0-44a8-8ac4-d8b040f84f59 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:43.384113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd7a62-bc3b-45dc-b3a7-dea53e0e21ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:43.384781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd7a62-bc3b-45dc-b3a7-dea53e0e21ed None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:43.385060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd7a62-bc3b-45dc-b3a7-dea53e0e21ed None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:43.385333 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd7a62-bc3b-45dc-b3a7-dea53e0e21ed None admin] PATH_INFO: `/v3/projects/226905b77a6b41a9af28e4c148199a6b/users/473ffcd293ba4fbba34e7e697268c28b/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:43.387295 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd7a62-bc3b-45dc-b3a7-dea53e0e21ed None admin] RBAC: Authorizing `identity:create_grant(project_id=226905b77a6b41a9af28e4c148199a6b, user_id=473ffcd293ba4fbba34e7e697268c28b, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:43.388749 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd7a62-bc3b-45dc-b3a7-dea53e0e21ed None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:43.391954 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:43.405107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 556/1084] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:21:43 2022] PUT /identity/v3/projects/226905b77a6b41a9af28e4c148199a6b/users/473ffcd293ba4fbba34e7e697268c28b/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:21:44.184252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd623b47-9222-463c-b1f8-4fe92922fa12 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:44.184819 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd623b47-9222-463c-b1f8-4fe92922fa12 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:44.184819 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd623b47-9222-463c-b1f8-4fe92922fa12 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:44.186183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 529/1085] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:44.191748 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701bbf9-81b0-4e2f-a21b-00841337e44f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:44.192249 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701bbf9-81b0-4e2f-a21b-00841337e44f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:44.192368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701bbf9-81b0-4e2f-a21b-00841337e44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:44.211381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1701bbf9-81b0-4e2f-a21b-00841337e44f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:44.212642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1701bbf9-81b0-4e2f-a21b-00841337e44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:44.291913 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 557/1086] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:44.314920 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd623b47-9222-463c-b1f8-4fe92922fa12 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:44.409857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:44.428403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd623b47-9222-463c-b1f8-4fe92922fa12 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:44.463219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ecedf-c495-41e6-a413-abe69dfeaa73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:44.463952 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ecedf-c495-41e6-a413-abe69dfeaa73 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:44.464230 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ecedf-c495-41e6-a413-abe69dfeaa73 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:44.464526 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ecedf-c495-41e6-a413-abe69dfeaa73 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:44.475491 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ecedf-c495-41e6-a413-abe69dfeaa73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:21:44.481088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-906ecedf-c495-41e6-a413-abe69dfeaa73 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:21:44.481088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:44.481088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:44.481088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:44.481088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:44.481088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:44.481088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:44.481088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:44.481088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:44.481088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:44.481088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:44.481088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:44.481088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:21:44.481088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:21:44.481088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:21:44.481088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:21:44.482218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:21:44.482218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:21:44.482218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:21:44.482218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:21:44.482218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:44.482218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:44.482218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:44.482218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:44.482218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:44.482218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:44.482218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:44.482218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:44.482218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:44.482218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:44.482218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:44.482218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:44.483500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:44.483500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:44.483500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:44.483500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:44.483500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:21:44.483500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:21:44.483500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:21:44.483500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:21:44.483500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:21:44.483500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:21:44.483500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:21:44.483500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:44.483500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 530/1087] 10.222.0.83 () {56 vars in 1120 bytes} [Mon Aug 1 15:21:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:21:44.488259 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701bbf9-81b0-4e2f-a21b-00841337e44f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:44.588626 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701bbf9-81b0-4e2f-a21b-00841337e44f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:44.613034 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:44.624106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7323172-5eea-424f-a6cc-3c83b8f5533c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:44.624724 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7323172-5eea-424f-a6cc-3c83b8f5533c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:44.624997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7323172-5eea-424f-a6cc-3c83b8f5533c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:44.625265 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7323172-5eea-424f-a6cc-3c83b8f5533c None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:44.626001 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7323172-5eea-424f-a6cc-3c83b8f5533c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:44.626304 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7323172-5eea-424f-a6cc-3c83b8f5533c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:44.627657 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7323172-5eea-424f-a6cc-3c83b8f5533c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:44.636030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 558/1088] 10.222.0.83 () {56 vars in 1115 bytes} [Mon Aug 1 15:21:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:44.641874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ecedf-c495-41e6-a413-abe69dfeaa73 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:44.747309 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ecedf-c495-41e6-a413-abe69dfeaa73 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:44.780872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d11a1-c4a7-4f85-916b-c19f92b2f7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:44.781509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d11a1-c4a7-4f85-916b-c19f92b2f7cb None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:44.781789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d11a1-c4a7-4f85-916b-c19f92b2f7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:44.782091 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d11a1-c4a7-4f85-916b-c19f92b2f7cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:44.792424 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d11a1-c4a7-4f85-916b-c19f92b2f7cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:44.793878 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d11a1-c4a7-4f85-916b-c19f92b2f7cb None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:44.796329 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:44.809057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-217d11a1-c4a7-4f85-916b-c19f92b2f7cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:44.809057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:44.809057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:44.809057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:44.809057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:44.809057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:44.809057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:44.809057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:44.809057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:44.809057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:44.809057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:44.809057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:44.809057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:44.809057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:44.809057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:44.810364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:44.810364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:44.810364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:44.810364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:44.810364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:44.810364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:44.810364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:44.810364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:44.810364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:44.810364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:44.810364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:44.810364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:44.810364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:44.810364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:44.810364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:44.810364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:44.810364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:44.812154 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:44.812154 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:44.812154 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:44.812154 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:44.812154 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 531/1089] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:44.818912 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7323172-5eea-424f-a6cc-3c83b8f5533c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:44.920732 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7323172-5eea-424f-a6cc-3c83b8f5533c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:44.955661 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10bee0-365d-4655-8189-919ac2fe4665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:44.956281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10bee0-365d-4655-8189-919ac2fe4665 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:44.956559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10bee0-365d-4655-8189-919ac2fe4665 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:44.956829 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10bee0-365d-4655-8189-919ac2fe4665 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:44.957511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10bee0-365d-4655-8189-919ac2fe4665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:44.957818 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10bee0-365d-4655-8189-919ac2fe4665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:44.959203 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10bee0-365d-4655-8189-919ac2fe4665 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:44.970977 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 559/1090] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:44.976827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d11a1-c4a7-4f85-916b-c19f92b2f7cb None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:45.077836 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d11a1-c4a7-4f85-916b-c19f92b2f7cb None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:45.110909 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d967d-48f9-4b6a-b7e3-2eb0253a02ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:45.111558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d967d-48f9-4b6a-b7e3-2eb0253a02ba None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:45.111840 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d967d-48f9-4b6a-b7e3-2eb0253a02ba None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:45.112115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d967d-48f9-4b6a-b7e3-2eb0253a02ba None admin] PATH_INFO: `/v3/users/nova` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:45.120350 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d967d-48f9-4b6a-b7e3-2eb0253a02ba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:45.121809 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d967d-48f9-4b6a-b7e3-2eb0253a02ba None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:45.134131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-592d967d-48f9-4b6a-b7e3-2eb0253a02ba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:21:45.134131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:45.134131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:45.134131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:45.134131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:45.134131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:45.134131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:45.134131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:45.134131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:45.134131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:45.134131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:45.134131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:45.134131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:21:45.134131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:21:45.134131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:21:45.134131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:21:45.135291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:45.135291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:45.135291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:21:45.135291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:45.135291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:21:45.135291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:21:45.135291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:45.135291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:45.135291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:45.135291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:45.135291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:45.135291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:45.135291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:45.135291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:45.135291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:45.135291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:45.135291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:45.135291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:45.136436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:45.136436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:45.136436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:21:45.136436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:21:45.136436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:21:45.136436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:21:45.136436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:21:45.136436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:21:45.136436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:21:45.136436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:45.136436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 532/1091] 10.222.0.83 () {56 vars in 1093 bytes} [Mon Aug 1 15:21:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:21:45.141025 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10bee0-365d-4655-8189-919ac2fe4665 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:45.192150 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:45.255930 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10bee0-365d-4655-8189-919ac2fe4665 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:45.289518 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4579c-f31d-4284-b83f-c3aff1a27e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:45.290247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4579c-f31d-4284-b83f-c3aff1a27e4b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:45.290526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4579c-f31d-4284-b83f-c3aff1a27e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:45.290793 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4579c-f31d-4284-b83f-c3aff1a27e4b None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:45.291580 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa4579c-f31d-4284-b83f-c3aff1a27e4b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:45.291886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa4579c-f31d-4284-b83f-c3aff1a27e4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:45.293308 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa4579c-f31d-4284-b83f-c3aff1a27e4b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:45.311299 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 560/1092] 10.222.0.83 () {56 vars in 1133 bytes} [Mon Aug 1 15:21:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:45.316998 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d967d-48f9-4b6a-b7e3-2eb0253a02ba None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:45.372786 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:45.416889 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d967d-48f9-4b6a-b7e3-2eb0253a02ba None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:45.430465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d54021-2449-46c2-9833-b3f4c541fdea None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:45.430750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d54021-2449-46c2-9833-b3f4c541fdea None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:45.431026 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d54021-2449-46c2-9833-b3f4c541fdea None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:45.432395 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 561/1093] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:21:45.437502 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26c708-d85e-408d-a05a-26536181b7e8 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:45.437792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26c708-d85e-408d-a05a-26536181b7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:45.438108 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26c708-d85e-408d-a05a-26536181b7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:45.454368 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3d33b-f2f9-46cf-bc5c-a0cb9416afff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:45.454922 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3d33b-f2f9-46cf-bc5c-a0cb9416afff None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:45.455204 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3d33b-f2f9-46cf-bc5c-a0cb9416afff None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:45.455502 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3d33b-f2f9-46cf-bc5c-a0cb9416afff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:45.455825 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-3f26c708-d85e-408d-a05a-26536181b7e8 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:45.457019 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3f26c708-d85e-408d-a05a-26536181b7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:45.526726 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3d33b-f2f9-46cf-bc5c-a0cb9416afff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:45.528204 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3d33b-f2f9-46cf-bc5c-a0cb9416afff None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:45.528341 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 562/1094] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:21:45.552038 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26c708-d85e-408d-a05a-26536181b7e8 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:45.587321 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:45.591780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:45.636153 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 533/1095] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:21:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:21:45.642980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3d33b-f2f9-46cf-bc5c-a0cb9416afff None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:45.664746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26c708-d85e-408d-a05a-26536181b7e8 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:45.698342 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1cd93-ee13-4961-9f0a-942aa8c73c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:45.698982 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1cd93-ee13-4961-9f0a-942aa8c73c70 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:45.699264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1cd93-ee13-4961-9f0a-942aa8c73c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:45.699538 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1cd93-ee13-4961-9f0a-942aa8c73c70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:45.709922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1cd93-ee13-4961-9f0a-942aa8c73c70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:21:45.715318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-9cd1cd93-ee13-4961-9f0a-942aa8c73c70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:21:45.715318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:45.715318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:45.715318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:45.715318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:45.715318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:45.715318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:45.715318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:45.715318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:45.715318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:45.715318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:45.715318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:45.715318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:21:45.715318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:21:45.715318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:21:45.715318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:21:45.716457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:21:45.716457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:21:45.716457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:21:45.716457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:21:45.716457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:45.716457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:45.716457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:45.716457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:45.716457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:45.716457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:45.716457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:45.716457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:45.716457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:45.716457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:45.716457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:45.716457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:45.717509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:45.717509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:45.717509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:45.717509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:45.717509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:21:45.717509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:21:45.717509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:21:45.717509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:21:45.717509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:21:45.717509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:21:45.717509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:21:45.717509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:21:45.721558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 563/1096] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:21:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:21:45.722395 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1cd93-ee13-4961-9f0a-942aa8c73c70 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:45.750984 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3d33b-f2f9-46cf-bc5c-a0cb9416afff None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:45.786521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2bd46-52d9-44b5-b9cc-ff097147e40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:45.787282 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2bd46-52d9-44b5-b9cc-ff097147e40e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:45.787567 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2bd46-52d9-44b5-b9cc-ff097147e40e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:45.787843 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2bd46-52d9-44b5-b9cc-ff097147e40e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:45.798925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2bd46-52d9-44b5-b9cc-ff097147e40e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:45.800382 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2bd46-52d9-44b5-b9cc-ff097147e40e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:45.816302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-bfd2bd46-52d9-44b5-b9cc-ff097147e40e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:45.816302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:45.816302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:45.816302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:45.816302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:45.816302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:45.816302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:45.816302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:45.816302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:45.816302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:45.816302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:45.816302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:45.816302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:21:45.816302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:21:45.816302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:21:45.817389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:21:45.817389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:45.817389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:45.817389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:45.817389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:45.817389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:45.817389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:45.817389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:45.817389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:45.817389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:45.817389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:45.817389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:45.817389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:45.817389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:45.817389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:45.817389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:45.817389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:45.818347 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:21:45.818347 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:21:45.818347 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:21:45.818347 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:45.818347 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 534/1097] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:21:45.821842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:45.823873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2bd46-52d9-44b5-b9cc-ff097147e40e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:45.828428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1cd93-ee13-4961-9f0a-942aa8c73c70 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:45.834852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:45.867297 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daabc2a-bc55-4b17-b808-1871d80af019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:45.867927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daabc2a-bc55-4b17-b808-1871d80af019 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:45.868310 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daabc2a-bc55-4b17-b808-1871d80af019 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:45.868489 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daabc2a-bc55-4b17-b808-1871d80af019 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:45.869211 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daabc2a-bc55-4b17-b808-1871d80af019 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:45.869519 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daabc2a-bc55-4b17-b808-1871d80af019 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:45.870936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daabc2a-bc55-4b17-b808-1871d80af019 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:45.879272 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 564/1098] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:21:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:45.884969 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daabc2a-bc55-4b17-b808-1871d80af019 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:45.929435 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2bd46-52d9-44b5-b9cc-ff097147e40e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:45.964304 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58661e28-24b0-40b8-ac05-0bd45ad787b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:45.964854 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58661e28-24b0-40b8-ac05-0bd45ad787b2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:45.965143 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58661e28-24b0-40b8-ac05-0bd45ad787b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:45.965419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58661e28-24b0-40b8-ac05-0bd45ad787b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:45.966141 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58661e28-24b0-40b8-ac05-0bd45ad787b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:45.966449 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58661e28-24b0-40b8-ac05-0bd45ad787b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:45.967829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58661e28-24b0-40b8-ac05-0bd45ad787b2 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:45.980948 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 535/1099] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:45.985296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daabc2a-bc55-4b17-b808-1871d80af019 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:45.986713 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58661e28-24b0-40b8-ac05-0bd45ad787b2 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:46.019414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1b7e9-c63f-4c8f-b488-0a1813e7dc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:46.020089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1b7e9-c63f-4c8f-b488-0a1813e7dc9f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:46.020368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1b7e9-c63f-4c8f-b488-0a1813e7dc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:46.020650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1b7e9-c63f-4c8f-b488-0a1813e7dc9f None admin] PATH_INFO: `/v3/users/473ffcd293ba4fbba34e7e697268c28b` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:46.021840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1b7e9-c63f-4c8f-b488-0a1813e7dc9f None admin] RBAC: Authorizing `identity:get_user(user_id=473ffcd293ba4fbba34e7e697268c28b)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:46.023918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1b7e9-c63f-4c8f-b488-0a1813e7dc9f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:46.025623 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 565/1100] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:21:45 2022] GET /identity/v3/users/473ffcd293ba4fbba34e7e697268c28b => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:46.031263 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1b7e9-c63f-4c8f-b488-0a1813e7dc9f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:46.053839 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:46.058723 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:46.089477 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58661e28-24b0-40b8-ac05-0bd45ad787b2 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:46.123572 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee861657-f027-4a68-bd94-a0187235c813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:46.124235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee861657-f027-4a68-bd94-a0187235c813 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:46.124544 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee861657-f027-4a68-bd94-a0187235c813 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:46.124821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee861657-f027-4a68-bd94-a0187235c813 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:46.132017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee861657-f027-4a68-bd94-a0187235c813 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:46.134240 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee861657-f027-4a68-bd94-a0187235c813 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:46.136115 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1b7e9-c63f-4c8f-b488-0a1813e7dc9f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:46.147848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-ee861657-f027-4a68-bd94-a0187235c813 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:46.147848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:21:46.147848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:21:46.147848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:21:46.147848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:21:46.147848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:21:46.147848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:21:46.147848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:21:46.147848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:21:46.147848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:21:46.147848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:21:46.147848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:21:46.147848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:21:46.147848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:21:46.147848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:21:46.149576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:21:46.149576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:21:46.149576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:21:46.149576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:21:46.149576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:21:46.149576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:21:46.149576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:21:46.149576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:21:46.149576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:21:46.149576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:21:46.149576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:21:46.149576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:21:46.149576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:21:46.149576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:21:46.149576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:21:46.149576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:21:46.149576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:21:46.150589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:21:46.150589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:21:46.150589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:21:46.150589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:21:46.150589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:21:46.150589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:21:46.150589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:21:46.150589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:21:46.150589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 536/1101] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:21:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:21:46.155693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee861657-f027-4a68-bd94-a0187235c813 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:46.174807 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2a68b-8c6c-4a93-aa97-1371bade423b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:46.175435 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2a68b-8c6c-4a93-aa97-1371bade423b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:46.175721 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2a68b-8c6c-4a93-aa97-1371bade423b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:46.175985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2a68b-8c6c-4a93-aa97-1371bade423b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:46.247882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2a68b-8c6c-4a93-aa97-1371bade423b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:46.249251 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2a68b-8c6c-4a93-aa97-1371bade423b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:46.259023 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee861657-f027-4a68-bd94-a0187235c813 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:46.277146 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:46.285939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:46.294304 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b6e14-c275-402a-ba8e-98e388b2e08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:46.294933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b6e14-c275-402a-ba8e-98e388b2e08c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:46.295286 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b6e14-c275-402a-ba8e-98e388b2e08c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:46.295645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b6e14-c275-402a-ba8e-98e388b2e08c None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:46.296320 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b6e14-c275-402a-ba8e-98e388b2e08c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:46.296632 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b6e14-c275-402a-ba8e-98e388b2e08c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:46.298102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b6e14-c275-402a-ba8e-98e388b2e08c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:46.310078 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 537/1102] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:21:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:46.316344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b6e14-c275-402a-ba8e-98e388b2e08c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:46.352074 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 566/1103] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:21:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:21:46.358177 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2a68b-8c6c-4a93-aa97-1371bade423b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:46.415331 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b6e14-c275-402a-ba8e-98e388b2e08c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:46.455720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4b4aa-6774-4d4f-b465-661182142e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:46.456348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4b4aa-6774-4d4f-b465-661182142e9d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:46.456634 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4b4aa-6774-4d4f-b465-661182142e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:46.456928 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4b4aa-6774-4d4f-b465-661182142e9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:46.457712 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4b4aa-6774-4d4f-b465-661182142e9d None admin] RBAC: Adding query filter params (role.id=37a24cea29194c7ca77701607e008914, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=25dc41ffc41c47fdbeec06110d04610c) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:46.458060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4b4aa-6774-4d4f-b465-661182142e9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:46.459505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4b4aa-6774-4d4f-b465-661182142e9d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:46.463860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2a68b-8c6c-4a93-aa97-1371bade423b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:46.465516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 538/1104] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:21:46 2022] GET /identity/v3/role_assignments?user.id=25dc41ffc41c47fdbeec06110d04610c&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=37a24cea29194c7ca77701607e008914 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:46.492219 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:46.502227 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e3375-e66b-465d-a025-faebed89bbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:46.502823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e3375-e66b-465d-a025-faebed89bbdd None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:46.503102 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e3375-e66b-465d-a025-faebed89bbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:46.503376 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e3375-e66b-465d-a025-faebed89bbdd None admin] PATH_INFO: `/v3/projects/226905b77a6b41a9af28e4c148199a6b` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:46.504636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e3375-e66b-465d-a025-faebed89bbdd None admin] RBAC: Authorizing `identity:get_project(project_id=226905b77a6b41a9af28e4c148199a6b)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:46.506145 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e3375-e66b-465d-a025-faebed89bbdd None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:46.508785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 567/1105] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:21:46 2022] GET /identity/v3/projects/226905b77a6b41a9af28e4c148199a6b => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:21:46.515359 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4b4aa-6774-4d4f-b465-661182142e9d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:46.574568 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:46.626171 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4b4aa-6774-4d4f-b465-661182142e9d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:46.663078 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e6900-3ab9-417b-84f7-29da164f1e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:46.663708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e6900-3ab9-417b-84f7-29da164f1e7e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:46.663994 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e6900-3ab9-417b-84f7-29da164f1e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:46.664266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e6900-3ab9-417b-84f7-29da164f1e7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:46.665048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e6900-3ab9-417b-84f7-29da164f1e7e None admin] RBAC: Adding query filter params (role.id=7bdfa43dc1d046f7965ffe52b74bc58f, scope.project.id=226905b77a6b41a9af28e4c148199a6b, user.id=473ffcd293ba4fbba34e7e697268c28b) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:46.665362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e6900-3ab9-417b-84f7-29da164f1e7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:46.666845 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e6900-3ab9-417b-84f7-29da164f1e7e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:46.672875 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 539/1106] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:21:46 2022] GET /identity/v3/role_assignments?user.id=473ffcd293ba4fbba34e7e697268c28b&scope.project.id=226905b77a6b41a9af28e4c148199a6b&role.id=7bdfa43dc1d046f7965ffe52b74bc58f => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:48.769585 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96515edc-ac0b-48c1-bf24-f70d729fb69d None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:48.770178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96515edc-ac0b-48c1-bf24-f70d729fb69d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:48.770178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96515edc-ac0b-48c1-bf24-f70d729fb69d None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:48.771523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 568/1107] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:48.777384 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce27420-f0ff-443e-bfb3-fb7d5b1f1475 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:48.777813 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce27420-f0ff-443e-bfb3-fb7d5b1f1475 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:48.777966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce27420-f0ff-443e-bfb3-fb7d5b1f1475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:48.796161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-0ce27420-f0ff-443e-bfb3-fb7d5b1f1475 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:48.797334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0ce27420-f0ff-443e-bfb3-fb7d5b1f1475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:48.874120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 540/1108] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:48.899894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96515edc-ac0b-48c1-bf24-f70d729fb69d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:48.999637 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96515edc-ac0b-48c1-bf24-f70d729fb69d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:49.033409 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cc981-ab2e-45e4-b104-642592ac3b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:49.034019 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cc981-ab2e-45e4-b104-642592ac3b04 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:49.034302 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cc981-ab2e-45e4-b104-642592ac3b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:49.034576 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cc981-ab2e-45e4-b104-642592ac3b04 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:49.035726 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cc981-ab2e-45e4-b104-642592ac3b04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:49.037152 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cc981-ab2e-45e4-b104-642592ac3b04 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:49.038799 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 569/1109] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:21:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:49.044627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce27420-f0ff-443e-bfb3-fb7d5b1f1475 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:49.090478 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:49.141835 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce27420-f0ff-443e-bfb3-fb7d5b1f1475 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:49.175635 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd44ba-d3dc-4305-a7c3-15f17de9f5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:49.176266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd44ba-d3dc-4305-a7c3-15f17de9f5ea None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:49.176804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd44ba-d3dc-4305-a7c3-15f17de9f5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:49.176804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd44ba-d3dc-4305-a7c3-15f17de9f5ea None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:49.177513 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd44ba-d3dc-4305-a7c3-15f17de9f5ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:49.179006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd44ba-d3dc-4305-a7c3-15f17de9f5ea None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:49.205121 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 541/1110] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Aug 1 15:21:49 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:21:51.303781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d88872-3a6b-412b-89e0-3c54394bb945 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:51.304332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d88872-3a6b-412b-89e0-3c54394bb945 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:51.304332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d88872-3a6b-412b-89e0-3c54394bb945 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:51.305651 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 570/1111] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:51.311396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e101ff-ee68-41bc-a3c2-9057a53d3bf9 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:51.311892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e101ff-ee68-41bc-a3c2-9057a53d3bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:51.312007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e101ff-ee68-41bc-a3c2-9057a53d3bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:51.332808 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-67e101ff-ee68-41bc-a3c2-9057a53d3bf9 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:51.332808 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-67e101ff-ee68-41bc-a3c2-9057a53d3bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:51.423893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 542/1112] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:51.449159 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d88872-3a6b-412b-89e0-3c54394bb945 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:51.521126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:51.566583 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d88872-3a6b-412b-89e0-3c54394bb945 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:51.601715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea28e1-793f-4a51-a99d-69769dcf9caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:51.602385 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea28e1-793f-4a51-a99d-69769dcf9caf None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:51.602662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea28e1-793f-4a51-a99d-69769dcf9caf None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:51.602929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea28e1-793f-4a51-a99d-69769dcf9caf None admin] PATH_INFO: `/v3/roles/73e29e4f51e6469facf94aaa698c5cfe` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:51.604370 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea28e1-793f-4a51-a99d-69769dcf9caf None admin] RBAC: Authorizing `identity:get_role(role_id=73e29e4f51e6469facf94aaa698c5cfe)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:51.605723 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea28e1-793f-4a51-a99d-69769dcf9caf None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:51.607110 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 571/1113] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:21:51 2022] GET /identity/v3/roles/73e29e4f51e6469facf94aaa698c5cfe => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:51.612977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e101ff-ee68-41bc-a3c2-9057a53d3bf9 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:51.647754 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:51.734726 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e101ff-ee68-41bc-a3c2-9057a53d3bf9 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:51.769727 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24a261-966a-46e3-961a-bca8cd52533f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:51.770409 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24a261-966a-46e3-961a-bca8cd52533f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:51.770691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24a261-966a-46e3-961a-bca8cd52533f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:51.770968 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24a261-966a-46e3-961a-bca8cd52533f None admin] PATH_INFO: `/v3/users/7af6ce7f83a3458e8277284a9ce06b9f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:51.784859 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24a261-966a-46e3-961a-bca8cd52533f None admin] RBAC: Authorizing `identity:get_user(user_id=7af6ce7f83a3458e8277284a9ce06b9f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:51.786504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24a261-966a-46e3-961a-bca8cd52533f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:51.788207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 543/1114] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:21:51 2022] GET /identity/v3/users/7af6ce7f83a3458e8277284a9ce06b9f => generated 286 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:51.794146 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea28e1-793f-4a51-a99d-69769dcf9caf None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:51.901786 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea28e1-793f-4a51-a99d-69769dcf9caf None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:51.930656 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:51.936848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32941ca5-9a06-41ce-9d39-75363041134e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:51.937473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32941ca5-9a06-41ce-9d39-75363041134e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:51.937757 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32941ca5-9a06-41ce-9d39-75363041134e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:51.938077 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32941ca5-9a06-41ce-9d39-75363041134e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:52.011063 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32941ca5-9a06-41ce-9d39-75363041134e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:52.012509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32941ca5-9a06-41ce-9d39-75363041134e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:52.112591 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 572/1115] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:21:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:21:52.119338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24a261-966a-46e3-961a-bca8cd52533f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:52.207995 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:52.217660 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24a261-966a-46e3-961a-bca8cd52533f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:52.252947 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bff950-87b4-4b5a-bea3-7acd1e9e33d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:52.253592 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bff950-87b4-4b5a-bea3-7acd1e9e33d5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:52.253873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bff950-87b4-4b5a-bea3-7acd1e9e33d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:52.254181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bff950-87b4-4b5a-bea3-7acd1e9e33d5 None admin] PATH_INFO: `/v3/projects/226905b77a6b41a9af28e4c148199a6b` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:52.255333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bff950-87b4-4b5a-bea3-7acd1e9e33d5 None admin] RBAC: Authorizing `identity:get_project(project_id=226905b77a6b41a9af28e4c148199a6b)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:52.256774 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bff950-87b4-4b5a-bea3-7acd1e9e33d5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:52.258511 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 544/1116] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:21:52 2022] GET /identity/v3/projects/226905b77a6b41a9af28e4c148199a6b => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:52.264739 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32941ca5-9a06-41ce-9d39-75363041134e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:52.295305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:52.378222 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32941ca5-9a06-41ce-9d39-75363041134e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:52.411240 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f274e2e-f80e-40b0-8ea3-0efebb4caa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:52.411844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f274e2e-f80e-40b0-8ea3-0efebb4caa0c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:52.412120 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f274e2e-f80e-40b0-8ea3-0efebb4caa0c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:52.412386 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f274e2e-f80e-40b0-8ea3-0efebb4caa0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:52.413237 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f274e2e-f80e-40b0-8ea3-0efebb4caa0c None admin] RBAC: Adding query filter params (role.id=73e29e4f51e6469facf94aaa698c5cfe, scope.project.id=226905b77a6b41a9af28e4c148199a6b, user.id=7af6ce7f83a3458e8277284a9ce06b9f) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:52.414157 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f274e2e-f80e-40b0-8ea3-0efebb4caa0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:52.415653 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f274e2e-f80e-40b0-8ea3-0efebb4caa0c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:52.420841 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 573/1117] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:21:52 2022] GET /identity/v3/role_assignments?user.id=7af6ce7f83a3458e8277284a9ce06b9f&scope.project.id=226905b77a6b41a9af28e4c148199a6b&role.id=73e29e4f51e6469facf94aaa698c5cfe => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:54.400943 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2425a9-6705-48b4-9c0f-2ce35b8ca51b None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:54.401468 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2425a9-6705-48b4-9c0f-2ce35b8ca51b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:54.401468 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2425a9-6705-48b4-9c0f-2ce35b8ca51b None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:54.402889 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 545/1118] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:54.408927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2cd31-47da-42ab-b220-626eaffa039d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:54.409366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2cd31-47da-42ab-b220-626eaffa039d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:54.409481 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2cd31-47da-42ab-b220-626eaffa039d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:54.427176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e8a2cd31-47da-42ab-b220-626eaffa039d None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:54.428353 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e8a2cd31-47da-42ab-b220-626eaffa039d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:54.493289 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 574/1119] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:54.515766 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2425a9-6705-48b4-9c0f-2ce35b8ca51b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:54.614935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2425a9-6705-48b4-9c0f-2ce35b8ca51b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:54.648062 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73080da5-fe1e-4826-8c2c-55f6ac6e6fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:54.648692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73080da5-fe1e-4826-8c2c-55f6ac6e6fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:54.648972 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73080da5-fe1e-4826-8c2c-55f6ac6e6fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:54.649241 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73080da5-fe1e-4826-8c2c-55f6ac6e6fd3 None admin] PATH_INFO: `/v3/roles/73e29e4f51e6469facf94aaa698c5cfe` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:54.650684 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73080da5-fe1e-4826-8c2c-55f6ac6e6fd3 None admin] RBAC: Authorizing `identity:get_role(role_id=73e29e4f51e6469facf94aaa698c5cfe)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:54.652110 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73080da5-fe1e-4826-8c2c-55f6ac6e6fd3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:54.653384 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 546/1120] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:21:54 2022] GET /identity/v3/roles/73e29e4f51e6469facf94aaa698c5cfe => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:54.659288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2cd31-47da-42ab-b220-626eaffa039d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:54.683393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:54.756232 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2cd31-47da-42ab-b220-626eaffa039d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:54.789080 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e3b8e-01d4-4aa8-a1b4-2261d62d402a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:54.789713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e3b8e-01d4-4aa8-a1b4-2261d62d402a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:54.790021 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e3b8e-01d4-4aa8-a1b4-2261d62d402a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:54.790293 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e3b8e-01d4-4aa8-a1b4-2261d62d402a None admin] PATH_INFO: `/v3/users/7af6ce7f83a3458e8277284a9ce06b9f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:54.791437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e3b8e-01d4-4aa8-a1b4-2261d62d402a None admin] RBAC: Authorizing `identity:get_user(user_id=7af6ce7f83a3458e8277284a9ce06b9f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:54.792862 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e3b8e-01d4-4aa8-a1b4-2261d62d402a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:54.794493 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 575/1121] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:21:54 2022] GET /identity/v3/users/7af6ce7f83a3458e8277284a9ce06b9f => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:54.800554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73080da5-fe1e-4826-8c2c-55f6ac6e6fd3 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:54.830670 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:54.903194 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73080da5-fe1e-4826-8c2c-55f6ac6e6fd3 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:54.936626 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45a4b1-b22c-4f2a-a0d9-94a34cf00a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:54.937265 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45a4b1-b22c-4f2a-a0d9-94a34cf00a14 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:54.937550 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45a4b1-b22c-4f2a-a0d9-94a34cf00a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:54.937827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45a4b1-b22c-4f2a-a0d9-94a34cf00a14 None admin] PATH_INFO: `/v3/projects/226905b77a6b41a9af28e4c148199a6b` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:54.939013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45a4b1-b22c-4f2a-a0d9-94a34cf00a14 None admin] RBAC: Authorizing `identity:get_project(project_id=226905b77a6b41a9af28e4c148199a6b)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:54.940452 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45a4b1-b22c-4f2a-a0d9-94a34cf00a14 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:54.942166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 547/1122] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:21:54 2022] GET /identity/v3/projects/226905b77a6b41a9af28e4c148199a6b => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:54.949308 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e3b8e-01d4-4aa8-a1b4-2261d62d402a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:55.036553 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:55.046178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e3b8e-01d4-4aa8-a1b4-2261d62d402a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:55.080696 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d5e95-14e3-4771-8c2b-d292bac075a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:55.081361 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d5e95-14e3-4771-8c2b-d292bac075a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:55.081646 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d5e95-14e3-4771-8c2b-d292bac075a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:55.081973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d5e95-14e3-4771-8c2b-d292bac075a5 None admin] PATH_INFO: `/v3/projects/226905b77a6b41a9af28e4c148199a6b/users/7af6ce7f83a3458e8277284a9ce06b9f/roles/73e29e4f51e6469facf94aaa698c5cfe` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:55.084154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d5e95-14e3-4771-8c2b-d292bac075a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=226905b77a6b41a9af28e4c148199a6b, user_id=7af6ce7f83a3458e8277284a9ce06b9f, role_id=73e29e4f51e6469facf94aaa698c5cfe)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:55.085593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d5e95-14e3-4771-8c2b-d292bac075a5 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:55.097987 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 576/1123] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:21:54 2022] PUT /identity/v3/projects/226905b77a6b41a9af28e4c148199a6b/users/7af6ce7f83a3458e8277284a9ce06b9f/roles/73e29e4f51e6469facf94aaa698c5cfe => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:21:57.067329 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05084abc-10ea-4a34-ab0f-7f523ccae2a5 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:57.067774 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05084abc-10ea-4a34-ab0f-7f523ccae2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:57.067869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05084abc-10ea-4a34-ab0f-7f523ccae2a5 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:57.069239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 548/1124] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:57.075201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04f37b-ae5a-42aa-b28d-a4bf4020976b None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:57.075601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04f37b-ae5a-42aa-b28d-a4bf4020976b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:57.075746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04f37b-ae5a-42aa-b28d-a4bf4020976b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:57.093364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2b04f37b-ae5a-42aa-b28d-a4bf4020976b None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:57.094555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2b04f37b-ae5a-42aa-b28d-a4bf4020976b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:57.159681 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 577/1125] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:21:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:21:57.183829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05084abc-10ea-4a34-ab0f-7f523ccae2a5 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:57.280116 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:57.288034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05084abc-10ea-4a34-ab0f-7f523ccae2a5 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:57.322921 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa8f43-69bc-49b3-a8e6-d471ec5272cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:57.323479 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa8f43-69bc-49b3-a8e6-d471ec5272cb None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:57.323757 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa8f43-69bc-49b3-a8e6-d471ec5272cb None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:57.324029 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa8f43-69bc-49b3-a8e6-d471ec5272cb None admin] PATH_INFO: `/v3/roles/73e29e4f51e6469facf94aaa698c5cfe` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:57.325481 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa8f43-69bc-49b3-a8e6-d471ec5272cb None admin] RBAC: Authorizing `identity:get_role(role_id=73e29e4f51e6469facf94aaa698c5cfe)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:57.326879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa8f43-69bc-49b3-a8e6-d471ec5272cb None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:57.328236 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 549/1126] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:21:57 2022] GET /identity/v3/roles/73e29e4f51e6469facf94aaa698c5cfe => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:57.334607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04f37b-ae5a-42aa-b28d-a4bf4020976b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:57.412880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:57.433305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04f37b-ae5a-42aa-b28d-a4bf4020976b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:57.470538 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8010e-f0b9-41ca-89f8-086532e5573e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:57.471185 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8010e-f0b9-41ca-89f8-086532e5573e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:57.471469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8010e-f0b9-41ca-89f8-086532e5573e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:57.471741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8010e-f0b9-41ca-89f8-086532e5573e None admin] PATH_INFO: `/v3/users/7af6ce7f83a3458e8277284a9ce06b9f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:57.472933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8010e-f0b9-41ca-89f8-086532e5573e None admin] RBAC: Authorizing `identity:get_user(user_id=7af6ce7f83a3458e8277284a9ce06b9f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:57.474463 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8010e-f0b9-41ca-89f8-086532e5573e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:57.476897 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 578/1127] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:21:57 2022] GET /identity/v3/users/7af6ce7f83a3458e8277284a9ce06b9f => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:57.483139 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa8f43-69bc-49b3-a8e6-d471ec5272cb None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:57.582559 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa8f43-69bc-49b3-a8e6-d471ec5272cb None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:57.616648 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9d43e-10ff-47b7-91af-ccef6e776679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:57.617264 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9d43e-10ff-47b7-91af-ccef6e776679 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:57.617548 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9d43e-10ff-47b7-91af-ccef6e776679 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:57.617828 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9d43e-10ff-47b7-91af-ccef6e776679 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:57.646513 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:57.684740 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9d43e-10ff-47b7-91af-ccef6e776679 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:57.686237 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9d43e-10ff-47b7-91af-ccef6e776679 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:57.785317 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 550/1128] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:21:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:21:57.791852 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8010e-f0b9-41ca-89f8-086532e5573e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:57.892492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8010e-f0b9-41ca-89f8-086532e5573e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:57.926657 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152bcf0-376f-4765-a626-1a88503582a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:57.927294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152bcf0-376f-4765-a626-1a88503582a0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:57.927578 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152bcf0-376f-4765-a626-1a88503582a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:57.927854 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152bcf0-376f-4765-a626-1a88503582a0 None admin] PATH_INFO: `/v3/projects/226905b77a6b41a9af28e4c148199a6b` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:57.929029 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1152bcf0-376f-4765-a626-1a88503582a0 None admin] RBAC: Authorizing `identity:get_project(project_id=226905b77a6b41a9af28e4c148199a6b)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:57.930492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1152bcf0-376f-4765-a626-1a88503582a0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:57.932163 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 579/1129] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:21:57 2022] GET /identity/v3/projects/226905b77a6b41a9af28e4c148199a6b => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:57.938378 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9d43e-10ff-47b7-91af-ccef6e776679 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:57.998523 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:58.036431 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9d43e-10ff-47b7-91af-ccef6e776679 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:58.068823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568eb79b-fa52-47a2-bd82-a75246677034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:58.069441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568eb79b-fa52-47a2-bd82-a75246677034 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:58.070677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568eb79b-fa52-47a2-bd82-a75246677034 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:58.070878 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568eb79b-fa52-47a2-bd82-a75246677034 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:58.071919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568eb79b-fa52-47a2-bd82-a75246677034 None admin] RBAC: Adding query filter params (role.id=73e29e4f51e6469facf94aaa698c5cfe, scope.project.id=226905b77a6b41a9af28e4c148199a6b, user.id=7af6ce7f83a3458e8277284a9ce06b9f) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:58.072225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568eb79b-fa52-47a2-bd82-a75246677034 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:58.073654 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568eb79b-fa52-47a2-bd82-a75246677034 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:58.079342 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 551/1130] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:21:57 2022] GET /identity/v3/role_assignments?user.id=7af6ce7f83a3458e8277284a9ce06b9f&scope.project.id=226905b77a6b41a9af28e4c148199a6b&role.id=73e29e4f51e6469facf94aaa698c5cfe => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:00.108292 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a582f7-7e11-4a9d-9625-ebee9118d3f6 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:00.108806 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a582f7-7e11-4a9d-9625-ebee9118d3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:00.108806 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a582f7-7e11-4a9d-9625-ebee9118d3f6 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:00.110201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 580/1131] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:22:00.115301 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7b488-2279-440d-af60-806f5d24bef5 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:00.115726 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7b488-2279-440d-af60-806f5d24bef5 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:00.115838 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7b488-2279-440d-af60-806f5d24bef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:00.134663 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7fe7b488-2279-440d-af60-806f5d24bef5 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:22:00.135840 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7fe7b488-2279-440d-af60-806f5d24bef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:00.201011 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 552/1132] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:22:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:22:00.224007 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a582f7-7e11-4a9d-9625-ebee9118d3f6 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:00.243981 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:00.324439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a582f7-7e11-4a9d-9625-ebee9118d3f6 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:00.357939 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6afda-2871-4ccf-839d-7e2860a59d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:00.358592 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6afda-2871-4ccf-839d-7e2860a59d9f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:00.358876 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6afda-2871-4ccf-839d-7e2860a59d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:00.359151 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6afda-2871-4ccf-839d-7e2860a59d9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:00.360327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6afda-2871-4ccf-839d-7e2860a59d9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:00.361768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6afda-2871-4ccf-839d-7e2860a59d9f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:00.363781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 581/1133] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:22:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:00.370835 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7b488-2279-440d-af60-806f5d24bef5 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:00.424431 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:00.467300 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7b488-2279-440d-af60-806f5d24bef5 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:00.501524 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0d744-45e0-4fd4-a497-dcbae026f54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:00.502163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0d744-45e0-4fd4-a497-dcbae026f54b None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:00.502444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0d744-45e0-4fd4-a497-dcbae026f54b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:00.502716 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0d744-45e0-4fd4-a497-dcbae026f54b None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:00.503398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0d744-45e0-4fd4-a497-dcbae026f54b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:00.504826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0d744-45e0-4fd4-a497-dcbae026f54b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:00.530535 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 553/1134] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Aug 1 15:22:00 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:22:02.599334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8debeab-6700-4573-888e-fd37d658ba82 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:02.599866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8debeab-6700-4573-888e-fd37d658ba82 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:02.599866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8debeab-6700-4573-888e-fd37d658ba82 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:02.601249 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 582/1135] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:22:02.606492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba239438-f1c3-4930-8c7d-cf8082645f6e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:02.606922 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba239438-f1c3-4930-8c7d-cf8082645f6e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:02.607055 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba239438-f1c3-4930-8c7d-cf8082645f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:02.625145 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ba239438-f1c3-4930-8c7d-cf8082645f6e None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:22:02.626380 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ba239438-f1c3-4930-8c7d-cf8082645f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:02.691495 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 554/1136] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:22:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:22:02.717636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8debeab-6700-4573-888e-fd37d658ba82 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:02.800898 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:02.816427 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8debeab-6700-4573-888e-fd37d658ba82 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:02.850880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60448c49-c767-4393-8d97-cd83f632c601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:02.851444 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60448c49-c767-4393-8d97-cd83f632c601 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:02.851725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60448c49-c767-4393-8d97-cd83f632c601 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:02.851997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60448c49-c767-4393-8d97-cd83f632c601 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:02.853137 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60448c49-c767-4393-8d97-cd83f632c601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:02.854592 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60448c49-c767-4393-8d97-cd83f632c601 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:02.856194 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 583/1137] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:22:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:02.862070 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba239438-f1c3-4930-8c7d-cf8082645f6e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:02.901297 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:02.961224 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba239438-f1c3-4930-8c7d-cf8082645f6e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:02.994250 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ebf97-df7b-4f41-a39d-5521e3ffc955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:02.994869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ebf97-df7b-4f41-a39d-5521e3ffc955 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:02.995151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ebf97-df7b-4f41-a39d-5521e3ffc955 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:02.995426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ebf97-df7b-4f41-a39d-5521e3ffc955 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:02.996103 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ebf97-df7b-4f41-a39d-5521e3ffc955 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:02.997541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ebf97-df7b-4f41-a39d-5521e3ffc955 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:03.019754 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 555/1138] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Aug 1 15:22:02 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:22:05.105920 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c48717-6da6-4a84-874c-024ba65472e5 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:05.106458 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c48717-6da6-4a84-874c-024ba65472e5 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:05.106458 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c48717-6da6-4a84-874c-024ba65472e5 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:05.107884 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 584/1139] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:22:05.113123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb780ff-0fa4-45d1-aec1-e837475a757d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:05.113555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb780ff-0fa4-45d1-aec1-e837475a757d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:05.113662 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb780ff-0fa4-45d1-aec1-e837475a757d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:05.131979 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ebb780ff-0fa4-45d1-aec1-e837475a757d None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:22:05.133186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ebb780ff-0fa4-45d1-aec1-e837475a757d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:05.200509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 556/1140] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:22:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:22:05.224486 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c48717-6da6-4a84-874c-024ba65472e5 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:05.336789 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c48717-6da6-4a84-874c-024ba65472e5 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:05.370291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23880866-cbc9-4550-b502-5570252f11bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:05.370919 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23880866-cbc9-4550-b502-5570252f11bb None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:05.371193 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23880866-cbc9-4550-b502-5570252f11bb None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:05.371460 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23880866-cbc9-4550-b502-5570252f11bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:05.381197 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23880866-cbc9-4550-b502-5570252f11bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:22:05.386234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-23880866-cbc9-4550-b502-5570252f11bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:22:05.386234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:22:05.386234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:22:05.386234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:22:05.386234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:22:05.386234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:22:05.386234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:22:05.386234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:22:05.386234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:22:05.386234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:22:05.386234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:22:05.386234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:22:05.386234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:22:05.386234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:22:05.386234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:22:05.386234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:22:05.387484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:22:05.387484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:22:05.387484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:22:05.387484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:22:05.387484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:22:05.387484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:22:05.387484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:22:05.387484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:22:05.387484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:22:05.387484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:22:05.387484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:22:05.387484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:22:05.387484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:22:05.387484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:22:05.387484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:22:05.387484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:22:05.388527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:22:05.388527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:22:05.388527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:22:05.388527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:22:05.388527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:22:05.388527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:22:05.388527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:22:05.388527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:22:05.388527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:22:05.388527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:22:05.388527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:22:05.388527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:22:05.388527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 585/1141] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:22:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:22:05.393465 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb780ff-0fa4-45d1-aec1-e837475a757d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:05.418506 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:05.492710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb780ff-0fa4-45d1-aec1-e837475a757d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:05.526260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed78a6c-d2c5-4a13-8961-6bbda637b9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:05.526891 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed78a6c-d2c5-4a13-8961-6bbda637b9db None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:05.527174 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed78a6c-d2c5-4a13-8961-6bbda637b9db None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:05.527450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed78a6c-d2c5-4a13-8961-6bbda637b9db None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:05.528165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed78a6c-d2c5-4a13-8961-6bbda637b9db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:22:05.528471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed78a6c-d2c5-4a13-8961-6bbda637b9db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:05.529935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed78a6c-d2c5-4a13-8961-6bbda637b9db None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:05.539327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 557/1142] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:22:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:05.545398 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23880866-cbc9-4550-b502-5570252f11bb None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:05.561629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:05.658284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23880866-cbc9-4550-b502-5570252f11bb None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:05.691730 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b09bec-d62a-4026-b9af-05b834ed051b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:05.692364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b09bec-d62a-4026-b9af-05b834ed051b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:05.692649 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b09bec-d62a-4026-b9af-05b834ed051b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:05.692937 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b09bec-d62a-4026-b9af-05b834ed051b None admin] PATH_INFO: `/v3/users/168acd4ffa894ab79a988b4432f85255` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:05.706987 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b09bec-d62a-4026-b9af-05b834ed051b None admin] RBAC: Authorizing `identity:get_user(user_id=168acd4ffa894ab79a988b4432f85255)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:05.708462 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b09bec-d62a-4026-b9af-05b834ed051b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:05.710807 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 586/1143] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:22:05 2022] GET /identity/v3/users/168acd4ffa894ab79a988b4432f85255 => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:05.716828 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed78a6c-d2c5-4a13-8961-6bbda637b9db None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:05.806529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:05.822705 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed78a6c-d2c5-4a13-8961-6bbda637b9db None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:05.856317 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58e66d-2d59-4a54-9fa8-563ecd12906c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:05.856942 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58e66d-2d59-4a54-9fa8-563ecd12906c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:05.857227 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58e66d-2d59-4a54-9fa8-563ecd12906c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:05.857505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58e66d-2d59-4a54-9fa8-563ecd12906c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:05.926771 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df58e66d-2d59-4a54-9fa8-563ecd12906c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:05.928153 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df58e66d-2d59-4a54-9fa8-563ecd12906c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:06.011506 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:06.027351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 558/1144] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:22:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:22:06.033302 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b09bec-d62a-4026-b9af-05b834ed051b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:06.102868 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:06.131862 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b09bec-d62a-4026-b9af-05b834ed051b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:06.167310 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb3393-874a-40b1-869a-4644c5654ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:06.167955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb3393-874a-40b1-869a-4644c5654ddc None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:06.168234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb3393-874a-40b1-869a-4644c5654ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:06.168505 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb3393-874a-40b1-869a-4644c5654ddc None admin] PATH_INFO: `/v3/projects/7993fb1ee84d4823adec6a165069d68c` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:06.169648 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb3393-874a-40b1-869a-4644c5654ddc None admin] RBAC: Authorizing `identity:get_project(project_id=7993fb1ee84d4823adec6a165069d68c)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:06.171115 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb3393-874a-40b1-869a-4644c5654ddc None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:06.172785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 587/1145] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:22:06 2022] GET /identity/v3/projects/7993fb1ee84d4823adec6a165069d68c => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:06.179583 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58e66d-2d59-4a54-9fa8-563ecd12906c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:06.276557 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58e66d-2d59-4a54-9fa8-563ecd12906c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:06.310235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae344f-c60e-444b-b917-d2182faddc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:06.310850 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae344f-c60e-444b-b917-d2182faddc2c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:06.311137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae344f-c60e-444b-b917-d2182faddc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:06.311417 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae344f-c60e-444b-b917-d2182faddc2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:06.312201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae344f-c60e-444b-b917-d2182faddc2c None admin] RBAC: Adding query filter params (role.id=7bdfa43dc1d046f7965ffe52b74bc58f, scope.project.id=7993fb1ee84d4823adec6a165069d68c, user.id=168acd4ffa894ab79a988b4432f85255) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:22:06.312513 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae344f-c60e-444b-b917-d2182faddc2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:06.313982 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae344f-c60e-444b-b917-d2182faddc2c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:06.319313 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 559/1146] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:22:06 2022] GET /identity/v3/role_assignments?user.id=168acd4ffa894ab79a988b4432f85255&scope.project.id=7993fb1ee84d4823adec6a165069d68c&role.id=7bdfa43dc1d046f7965ffe52b74bc58f => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:08.326389 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7720d2b-e317-4449-84ee-f6d0f2b77cb1 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:08.326885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7720d2b-e317-4449-84ee-f6d0f2b77cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:08.326885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7720d2b-e317-4449-84ee-f6d0f2b77cb1 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:08.328200 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 588/1147] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:22:08.333383 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2108e1-3c40-4255-a0f1-22fb9658315b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:08.333772 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2108e1-3c40-4255-a0f1-22fb9658315b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:08.333968 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2108e1-3c40-4255-a0f1-22fb9658315b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:08.352824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3b2108e1-3c40-4255-a0f1-22fb9658315b None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:22:08.354060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3b2108e1-3c40-4255-a0f1-22fb9658315b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:08.400726 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:08.422274 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 560/1148] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:22:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:22:08.445399 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7720d2b-e317-4449-84ee-f6d0f2b77cb1 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:08.545430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7720d2b-e317-4449-84ee-f6d0f2b77cb1 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:08.578849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330914df-9598-483f-aa3f-11a7cef7a63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:08.579416 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330914df-9598-483f-aa3f-11a7cef7a63f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:08.579696 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330914df-9598-483f-aa3f-11a7cef7a63f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:08.579968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330914df-9598-483f-aa3f-11a7cef7a63f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:08.588074 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:08.591453 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330914df-9598-483f-aa3f-11a7cef7a63f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:22:08.596562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-330914df-9598-483f-aa3f-11a7cef7a63f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:22:08.596562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:22:08.596562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:22:08.596562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:22:08.596562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:22:08.596562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:22:08.596562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:22:08.596562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:22:08.596562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:22:08.596562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:22:08.596562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:22:08.596562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:22:08.596562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:22:08.596562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:22:08.596562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:22:08.596562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:22:08.597652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:22:08.597652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:22:08.597652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:22:08.597652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:22:08.597652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:22:08.597652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:22:08.597652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:22:08.597652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:22:08.597652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:22:08.597652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:22:08.597652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:22:08.597652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:22:08.597652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:22:08.597652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:22:08.597652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:22:08.597652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:22:08.598922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:22:08.598922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:22:08.598922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:22:08.598922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:22:08.598922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:22:08.598922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:22:08.598922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:22:08.598922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:22:08.598922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:22:08.598922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:22:08.598922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:22:08.598922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:22:08.598922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 589/1149] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:22:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:22:08.603978 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2108e1-3c40-4255-a0f1-22fb9658315b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:08.701789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2108e1-3c40-4255-a0f1-22fb9658315b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:08.736047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5cd76-bd2e-48d2-b920-6027aef55328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:08.736670 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5cd76-bd2e-48d2-b920-6027aef55328 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:08.736949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5cd76-bd2e-48d2-b920-6027aef55328 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:08.737223 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5cd76-bd2e-48d2-b920-6027aef55328 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:08.737958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5cd76-bd2e-48d2-b920-6027aef55328 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:22:08.738268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5cd76-bd2e-48d2-b920-6027aef55328 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:08.739644 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5cd76-bd2e-48d2-b920-6027aef55328 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:08.747599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 561/1150] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:22:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:08.753449 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330914df-9598-483f-aa3f-11a7cef7a63f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:08.852180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330914df-9598-483f-aa3f-11a7cef7a63f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:08.886157 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30a961-610b-4db4-b28e-915731701477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:08.886784 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30a961-610b-4db4-b28e-915731701477 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:08.887061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30a961-610b-4db4-b28e-915731701477 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:08.887332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30a961-610b-4db4-b28e-915731701477 None admin] PATH_INFO: `/v3/users/168acd4ffa894ab79a988b4432f85255` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:08.888509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d30a961-610b-4db4-b28e-915731701477 None admin] RBAC: Authorizing `identity:get_user(user_id=168acd4ffa894ab79a988b4432f85255)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:08.889964 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d30a961-610b-4db4-b28e-915731701477 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:08.891583 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 590/1151] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:22:08 2022] GET /identity/v3/users/168acd4ffa894ab79a988b4432f85255 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:08.897713 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5cd76-bd2e-48d2-b920-6027aef55328 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:08.940474 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:08.996641 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5cd76-bd2e-48d2-b920-6027aef55328 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:09.028989 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdae5c-4f73-4f54-bbc9-8a7bc5d22021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:09.029621 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdae5c-4f73-4f54-bbc9-8a7bc5d22021 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:09.029921 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdae5c-4f73-4f54-bbc9-8a7bc5d22021 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:09.030198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdae5c-4f73-4f54-bbc9-8a7bc5d22021 None admin] PATH_INFO: `/v3/projects/7993fb1ee84d4823adec6a165069d68c` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:09.031331 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cdae5c-4f73-4f54-bbc9-8a7bc5d22021 None admin] RBAC: Authorizing `identity:get_project(project_id=7993fb1ee84d4823adec6a165069d68c)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:09.032764 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cdae5c-4f73-4f54-bbc9-8a7bc5d22021 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:09.034450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 562/1152] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:22:08 2022] GET /identity/v3/projects/7993fb1ee84d4823adec6a165069d68c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:09.040474 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30a961-610b-4db4-b28e-915731701477 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:09.106205 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:09.138576 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30a961-610b-4db4-b28e-915731701477 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:09.170691 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e0777-a47f-4a72-8344-c69a3c58773a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:09.171351 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e0777-a47f-4a72-8344-c69a3c58773a None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:09.171634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e0777-a47f-4a72-8344-c69a3c58773a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:09.171909 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e0777-a47f-4a72-8344-c69a3c58773a None admin] PATH_INFO: `/v3/projects/7993fb1ee84d4823adec6a165069d68c/users/168acd4ffa894ab79a988b4432f85255/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:09.173855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e0777-a47f-4a72-8344-c69a3c58773a None admin] RBAC: Authorizing `identity:create_grant(project_id=7993fb1ee84d4823adec6a165069d68c, user_id=168acd4ffa894ab79a988b4432f85255, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:09.175319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e0777-a47f-4a72-8344-c69a3c58773a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:09.189314 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 591/1153] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:22:09 2022] PUT /identity/v3/projects/7993fb1ee84d4823adec6a165069d68c/users/168acd4ffa894ab79a988b4432f85255/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:22:11.184759 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739adce-7356-4516-9f0f-868954cc4c02 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:11.185312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739adce-7356-4516-9f0f-868954cc4c02 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:11.185312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739adce-7356-4516-9f0f-868954cc4c02 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:11.186711 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 563/1154] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:22:11.192065 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3638f5-0963-4e55-927e-22e7ae475533 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:11.192509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3638f5-0963-4e55-927e-22e7ae475533 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:11.192615 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3638f5-0963-4e55-927e-22e7ae475533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:11.211043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8f3638f5-0963-4e55-927e-22e7ae475533 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:22:11.212210 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8f3638f5-0963-4e55-927e-22e7ae475533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:11.276330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 592/1155] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:22:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:22:11.299551 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739adce-7356-4516-9f0f-868954cc4c02 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:11.402460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739adce-7356-4516-9f0f-868954cc4c02 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:11.412405 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:11.435930 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd47fe0-1a97-4bed-94da-e28185f64c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:11.436574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd47fe0-1a97-4bed-94da-e28185f64c4d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:11.436857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd47fe0-1a97-4bed-94da-e28185f64c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:11.437135 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd47fe0-1a97-4bed-94da-e28185f64c4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:11.448853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd47fe0-1a97-4bed-94da-e28185f64c4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:22:11.453795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-1bd47fe0-1a97-4bed-94da-e28185f64c4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:22:11.453795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:22:11.453795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:22:11.453795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:22:11.453795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:22:11.453795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:22:11.453795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:22:11.453795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:22:11.453795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:22:11.453795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:22:11.453795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:22:11.453795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:22:11.453795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:22:11.453795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:22:11.453795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:22:11.453795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:22:11.454944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:22:11.454944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:22:11.454944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:22:11.454944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:22:11.454944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:22:11.454944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:22:11.454944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:22:11.454944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:22:11.454944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:22:11.454944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:22:11.454944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:22:11.454944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:22:11.454944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:22:11.454944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:22:11.454944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:22:11.454944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:22:11.456044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:22:11.456044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:22:11.456044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:22:11.456044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:22:11.456044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:22:11.456044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:22:11.456044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:22:11.456044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:22:11.456044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:22:11.456044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:22:11.456044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:22:11.456044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:22:11.456044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 564/1156] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:22:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:22:11.461431 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3638f5-0963-4e55-927e-22e7ae475533 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:11.521742 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:11.563770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3638f5-0963-4e55-927e-22e7ae475533 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:11.597650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b39791-b988-4b3b-a578-fe517d3f6536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:11.598290 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b39791-b988-4b3b-a578-fe517d3f6536 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:11.598571 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b39791-b988-4b3b-a578-fe517d3f6536 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:11.599061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b39791-b988-4b3b-a578-fe517d3f6536 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:11.599680 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b39791-b988-4b3b-a578-fe517d3f6536 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:22:11.599981 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b39791-b988-4b3b-a578-fe517d3f6536 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:11.601870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b39791-b988-4b3b-a578-fe517d3f6536 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:11.609678 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 593/1157] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:22:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:11.615536 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd47fe0-1a97-4bed-94da-e28185f64c4d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:11.715547 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd47fe0-1a97-4bed-94da-e28185f64c4d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:11.751314 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ace5dc-c6aa-47f4-8f80-96ce0adddc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:11.751965 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ace5dc-c6aa-47f4-8f80-96ce0adddc8b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:11.752253 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ace5dc-c6aa-47f4-8f80-96ce0adddc8b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:11.752526 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ace5dc-c6aa-47f4-8f80-96ce0adddc8b None admin] PATH_INFO: `/v3/users/168acd4ffa894ab79a988b4432f85255` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:11.753693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ace5dc-c6aa-47f4-8f80-96ce0adddc8b None admin] RBAC: Authorizing `identity:get_user(user_id=168acd4ffa894ab79a988b4432f85255)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:11.755166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ace5dc-c6aa-47f4-8f80-96ce0adddc8b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:11.756764 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 565/1158] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:22:11 2022] GET /identity/v3/users/168acd4ffa894ab79a988b4432f85255 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:11.762723 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b39791-b988-4b3b-a578-fe517d3f6536 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:11.862510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b39791-b988-4b3b-a578-fe517d3f6536 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:11.881341 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:11.898622 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21828b-a67b-4033-9db5-00bd68362bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:11.899245 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21828b-a67b-4033-9db5-00bd68362bbd None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:11.899524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21828b-a67b-4033-9db5-00bd68362bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:11.899793 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21828b-a67b-4033-9db5-00bd68362bbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:11.968104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21828b-a67b-4033-9db5-00bd68362bbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:11.969642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21828b-a67b-4033-9db5-00bd68362bbd None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:12.066671 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 594/1159] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:22:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:22:12.072840 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ace5dc-c6aa-47f4-8f80-96ce0adddc8b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:12.116616 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:12.183806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ace5dc-c6aa-47f4-8f80-96ce0adddc8b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:12.217970 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e844bd-0145-4801-ac94-6168b7db8c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:12.218655 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e844bd-0145-4801-ac94-6168b7db8c7a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:12.218863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e844bd-0145-4801-ac94-6168b7db8c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:12.219144 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e844bd-0145-4801-ac94-6168b7db8c7a None admin] PATH_INFO: `/v3/projects/7993fb1ee84d4823adec6a165069d68c` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:12.220297 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e844bd-0145-4801-ac94-6168b7db8c7a None admin] RBAC: Authorizing `identity:get_project(project_id=7993fb1ee84d4823adec6a165069d68c)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:12.221743 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e844bd-0145-4801-ac94-6168b7db8c7a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:12.223436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 566/1160] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:22:12 2022] GET /identity/v3/projects/7993fb1ee84d4823adec6a165069d68c => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:12.230588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21828b-a67b-4033-9db5-00bd68362bbd None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:12.251243 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:12.337241 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21828b-a67b-4033-9db5-00bd68362bbd None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:12.369674 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8aee0e-5310-4c6f-ba44-8ed86264f929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:12.370306 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8aee0e-5310-4c6f-ba44-8ed86264f929 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:12.370589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8aee0e-5310-4c6f-ba44-8ed86264f929 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:12.370863 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8aee0e-5310-4c6f-ba44-8ed86264f929 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:12.371635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8aee0e-5310-4c6f-ba44-8ed86264f929 None admin] RBAC: Adding query filter params (role.id=7bdfa43dc1d046f7965ffe52b74bc58f, scope.project.id=7993fb1ee84d4823adec6a165069d68c, user.id=168acd4ffa894ab79a988b4432f85255) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:22:12.371946 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8aee0e-5310-4c6f-ba44-8ed86264f929 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:12.373393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8aee0e-5310-4c6f-ba44-8ed86264f929 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:12.379037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 595/1161] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:22:12 2022] GET /identity/v3/role_assignments?user.id=168acd4ffa894ab79a988b4432f85255&scope.project.id=7993fb1ee84d4823adec6a165069d68c&role.id=7bdfa43dc1d046f7965ffe52b74bc58f => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:14.407040 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de0c90-89b2-4ffb-8d18-7c85b282e131 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:14.407646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de0c90-89b2-4ffb-8d18-7c85b282e131 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:14.407646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de0c90-89b2-4ffb-8d18-7c85b282e131 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:14.408949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 567/1162] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:22:14.414370 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13ab27-d6a8-4d65-8f9a-5a976407e24d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:14.414805 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13ab27-d6a8-4d65-8f9a-5a976407e24d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:14.414911 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13ab27-d6a8-4d65-8f9a-5a976407e24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:14.433013 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8f13ab27-d6a8-4d65-8f9a-5a976407e24d None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:22:14.434223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8f13ab27-d6a8-4d65-8f9a-5a976407e24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:14.500000 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 596/1163] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:22:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:22:14.521986 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de0c90-89b2-4ffb-8d18-7c85b282e131 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:14.633020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de0c90-89b2-4ffb-8d18-7c85b282e131 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:14.640280 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:14.670249 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc099c3e-8158-49bf-b46a-e5e669d8d1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:14.670800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc099c3e-8158-49bf-b46a-e5e669d8d1fd None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:14.671083 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc099c3e-8158-49bf-b46a-e5e669d8d1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:14.671356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc099c3e-8158-49bf-b46a-e5e669d8d1fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:14.742437 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc099c3e-8158-49bf-b46a-e5e669d8d1fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:14.743904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc099c3e-8158-49bf-b46a-e5e669d8d1fd None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:14.847410 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 568/1164] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:22:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:22:14.854215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13ab27-d6a8-4d65-8f9a-5a976407e24d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:14.868796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:14.953338 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13ab27-d6a8-4d65-8f9a-5a976407e24d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:14.987167 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d787ad-b5bb-472d-b1bb-bee535f26cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:14.987823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d787ad-b5bb-472d-b1bb-bee535f26cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:14.988107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d787ad-b5bb-472d-b1bb-bee535f26cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:14.988382 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d787ad-b5bb-472d-b1bb-bee535f26cb4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:14.995820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d787ad-b5bb-472d-b1bb-bee535f26cb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:14.997300 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d787ad-b5bb-472d-b1bb-bee535f26cb4 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:15.010060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-37d787ad-b5bb-472d-b1bb-bee535f26cb4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 15:22:15.010060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:22:15.010060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 01 15:22:15.010060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 01 15:22:15.010060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:22:15.010060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:22:15.010060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:22:15.010060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:22:15.010060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 01 15:22:15.010060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:22:15.010060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:22:15.010060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:22:15.010060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:22:15.010060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:22:15.010060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:22:15.010060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:22:15.010060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:22:15.011488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:22:15.011488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:22:15.011488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:22:15.011488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:22:15.011488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:22:15.011488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:22:15.011488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:22:15.011488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:22:15.011488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:22:15.011488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:22:15.011488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:22:15.011488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:22:15.011488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:22:15.011488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:22:15.011488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:22:15.011488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:22:15.012921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:22:15.012921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:22:15.012921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:22:15.012921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:22:15.012921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:22:15.012921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:22:15.012921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:22:15.012921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:22:15.012921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:22:15.012921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:22:15.012921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 01 15:22:15.012921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:22:15.012921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 15:22:15.012921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:22:15.012921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 597/1165] 10.222.0.83 () {56 vars in 1117 bytes} [Mon Aug 1 15:22:14 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:22:15.017396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc099c3e-8158-49bf-b46a-e5e669d8d1fd None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:15.026046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:15.118326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc099c3e-8158-49bf-b46a-e5e669d8d1fd None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:15.152357 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a33bfc-b4f6-4dd7-89a5-05a95746edd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:15.152978 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a33bfc-b4f6-4dd7-89a5-05a95746edd6 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:15.153264 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a33bfc-b4f6-4dd7-89a5-05a95746edd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:15.153541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a33bfc-b4f6-4dd7-89a5-05a95746edd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:15.154260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a33bfc-b4f6-4dd7-89a5-05a95746edd6 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:22:15.154569 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a33bfc-b4f6-4dd7-89a5-05a95746edd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:15.155947 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a33bfc-b4f6-4dd7-89a5-05a95746edd6 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:15.163049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 569/1166] 10.222.0.83 () {56 vars in 1115 bytes} [Mon Aug 1 15:22:15 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:17.152333 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7ffdb-245e-420f-be74-68e16b9ef9bf None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:17.152884 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7ffdb-245e-420f-be74-68e16b9ef9bf None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:17.152884 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7ffdb-245e-420f-be74-68e16b9ef9bf None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:17.154268 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 598/1167] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:22:17.159533 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971e3dc-1429-4cd5-b377-95255395bf85 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:17.159956 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971e3dc-1429-4cd5-b377-95255395bf85 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:17.160058 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971e3dc-1429-4cd5-b377-95255395bf85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:17.178429 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-0971e3dc-1429-4cd5-b377-95255395bf85 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:22:17.179799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0971e3dc-1429-4cd5-b377-95255395bf85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:17.244804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:17.250091 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 570/1168] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:22:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:22:17.273119 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7ffdb-245e-420f-be74-68e16b9ef9bf None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:17.348913 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:17.374826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7ffdb-245e-420f-be74-68e16b9ef9bf None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:17.408335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db5ffe-0b0d-4d8d-98f2-43e8038ade04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:17.408965 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db5ffe-0b0d-4d8d-98f2-43e8038ade04 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:17.409242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db5ffe-0b0d-4d8d-98f2-43e8038ade04 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:17.409512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db5ffe-0b0d-4d8d-98f2-43e8038ade04 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:17.410243 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3db5ffe-0b0d-4d8d-98f2-43e8038ade04 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:17.411611 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3db5ffe-0b0d-4d8d-98f2-43e8038ade04 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:17.428281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 599/1169] 10.222.0.83 () {60 vars in 1138 bytes} [Mon Aug 1 15:22:17 2022] POST /identity/v3/domains => generated 267 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:22:19.470570 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd9dec-c357-4503-a7dd-80353e42ab4d None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:19.471134 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd9dec-c357-4503-a7dd-80353e42ab4d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:19.471134 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd9dec-c357-4503-a7dd-80353e42ab4d None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:19.472472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 571/1170] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:22:19.478598 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046c93b-3442-4337-9965-7baf9f27c1cf None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:19.479312 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046c93b-3442-4337-9965-7baf9f27c1cf None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:19.479523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046c93b-3442-4337-9965-7baf9f27c1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:19.497317 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-7046c93b-3442-4337-9965-7baf9f27c1cf None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:22:19.498568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7046c93b-3442-4337-9965-7baf9f27c1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:19.564892 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 600/1171] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:22:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:22:19.588793 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd9dec-c357-4503-a7dd-80353e42ab4d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:19.688983 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd9dec-c357-4503-a7dd-80353e42ab4d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:19.722888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1337a-1215-49c2-8cd8-88d4faf35eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:19.723374 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1337a-1215-49c2-8cd8-88d4faf35eae None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:19.723651 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1337a-1215-49c2-8cd8-88d4faf35eae None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:19.723922 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1337a-1215-49c2-8cd8-88d4faf35eae None admin] PATH_INFO: `/v3/domains/7758e7b1fe56410da8c7d1dc21b6f8e2` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:19.735731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f1337a-1215-49c2-8cd8-88d4faf35eae None admin] RBAC: Authorizing `identity:get_domain(domain_id=7758e7b1fe56410da8c7d1dc21b6f8e2)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:19.737195 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f1337a-1215-49c2-8cd8-88d4faf35eae None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:19.738930 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 572/1172] 10.222.0.83 () {56 vars in 1183 bytes} [Mon Aug 1 15:22:19 2022] GET /identity/v3/domains/7758e7b1fe56410da8c7d1dc21b6f8e2 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:19.745168 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046c93b-3442-4337-9965-7baf9f27c1cf None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:19.807234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:19.849179 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046c93b-3442-4337-9965-7baf9f27c1cf None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:19.883603 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58c8db-ad64-45c3-afb4-69358dfb3fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:19.884232 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58c8db-ad64-45c3-afb4-69358dfb3fdc None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:19.884512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58c8db-ad64-45c3-afb4-69358dfb3fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:19.884785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58c8db-ad64-45c3-afb4-69358dfb3fdc None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:19.885460 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58c8db-ad64-45c3-afb4-69358dfb3fdc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:19.886951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58c8db-ad64-45c3-afb4-69358dfb3fdc None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:19.914721 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 601/1173] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Aug 1 15:22:19 2022] POST /identity/v3/projects => generated 361 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:22:21.995699 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31251bc-a70c-4616-a84d-949b1df0abff None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:21.996265 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31251bc-a70c-4616-a84d-949b1df0abff None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:21.996265 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31251bc-a70c-4616-a84d-949b1df0abff None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:21.997612 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 573/1174] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:22:22.003002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f271f74-849a-43a5-a23a-08ec8b0e5b64 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:22.003406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f271f74-849a-43a5-a23a-08ec8b0e5b64 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:22.003565 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f271f74-849a-43a5-a23a-08ec8b0e5b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:22.021314 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8f271f74-849a-43a5-a23a-08ec8b0e5b64 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:22:22.022525 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8f271f74-849a-43a5-a23a-08ec8b0e5b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:22.090808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 602/1175] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:22:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:22:22.117567 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31251bc-a70c-4616-a84d-949b1df0abff None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:22.176928 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:22.223070 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31251bc-a70c-4616-a84d-949b1df0abff None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:22.261085 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf96c6a-fb81-4cc7-a3c6-3a5f2aedeb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:22.261638 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf96c6a-fb81-4cc7-a3c6-3a5f2aedeb83 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:22.261942 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf96c6a-fb81-4cc7-a3c6-3a5f2aedeb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:22.262218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf96c6a-fb81-4cc7-a3c6-3a5f2aedeb83 None admin] PATH_INFO: `/v3/domains/7758e7b1fe56410da8c7d1dc21b6f8e2` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:22.263359 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf96c6a-fb81-4cc7-a3c6-3a5f2aedeb83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7758e7b1fe56410da8c7d1dc21b6f8e2)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:22.265025 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf96c6a-fb81-4cc7-a3c6-3a5f2aedeb83 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:22.267008 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 574/1176] 10.222.0.83 () {56 vars in 1183 bytes} [Mon Aug 1 15:22:22 2022] GET /identity/v3/domains/7758e7b1fe56410da8c7d1dc21b6f8e2 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:22.272891 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f271f74-849a-43a5-a23a-08ec8b0e5b64 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:22.319392 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:22.374092 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f271f74-849a-43a5-a23a-08ec8b0e5b64 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:22.407745 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dc10d-76a3-4f11-a482-d8f418eb126b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:22.408361 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dc10d-76a3-4f11-a482-d8f418eb126b None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:22.408645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dc10d-76a3-4f11-a482-d8f418eb126b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:22.408918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dc10d-76a3-4f11-a482-d8f418eb126b None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:22.409598 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5dc10d-76a3-4f11-a482-d8f418eb126b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:22.411115 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5dc10d-76a3-4f11-a482-d8f418eb126b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:22.432867 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 603/1177] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Aug 1 15:22:22 2022] POST /identity/v3/users => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:22:24.477221 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2211f-711f-4384-8d68-afa7eaa7c9f7 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:24.477755 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2211f-711f-4384-8d68-afa7eaa7c9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:24.477755 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2211f-711f-4384-8d68-afa7eaa7c9f7 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:24.479160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 575/1178] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:22:24.484229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed15be-8b35-4784-ba4d-b2e07bd157cc None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:24.484624 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed15be-8b35-4784-ba4d-b2e07bd157cc None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:24.484773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed15be-8b35-4784-ba4d-b2e07bd157cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:24.502805 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e8ed15be-8b35-4784-ba4d-b2e07bd157cc None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:22:24.503971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e8ed15be-8b35-4784-ba4d-b2e07bd157cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:24.570371 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 604/1179] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:22:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:22:24.594706 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2211f-711f-4384-8d68-afa7eaa7c9f7 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:24.695878 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2211f-711f-4384-8d68-afa7eaa7c9f7 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:24.721490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:24.729939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654e9cd-b500-449b-b066-40b91f32ca52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:24.730536 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654e9cd-b500-449b-b066-40b91f32ca52 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:24.730820 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654e9cd-b500-449b-b066-40b91f32ca52 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:24.731094 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654e9cd-b500-449b-b066-40b91f32ca52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:24.742132 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654e9cd-b500-449b-b066-40b91f32ca52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:22:24.747201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-e654e9cd-b500-449b-b066-40b91f32ca52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:22:24.747201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:22:24.747201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:22:24.747201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:22:24.747201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:22:24.747201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:22:24.747201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:22:24.747201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:22:24.747201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:22:24.747201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:22:24.747201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:22:24.747201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:22:24.747201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:22:24.747201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:22:24.747201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:22:24.747201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:22:24.748356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:22:24.748356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:22:24.748356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:22:24.748356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:22:24.748356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:22:24.748356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:22:24.748356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:22:24.748356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:22:24.748356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:22:24.748356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:22:24.748356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:22:24.748356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:22:24.748356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:22:24.748356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:22:24.748356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:22:24.748356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:22:24.749425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:22:24.749425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:22:24.749425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:22:24.749425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:22:24.749425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:22:24.749425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:22:24.749425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:22:24.749425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:22:24.749425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:22:24.749425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:22:24.749425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:22:24.749425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:22:24.749425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 576/1180] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:22:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:22:24.754557 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed15be-8b35-4784-ba4d-b2e07bd157cc None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:24.784256 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:24.851981 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed15be-8b35-4784-ba4d-b2e07bd157cc None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:24.885679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b8b45-0a0f-4c35-812a-3566a92382e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:24.886321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b8b45-0a0f-4c35-812a-3566a92382e0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:24.886598 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b8b45-0a0f-4c35-812a-3566a92382e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:24.886865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b8b45-0a0f-4c35-812a-3566a92382e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:24.887565 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b8b45-0a0f-4c35-812a-3566a92382e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:22:24.891547 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b8b45-0a0f-4c35-812a-3566a92382e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:24.891547 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b8b45-0a0f-4c35-812a-3566a92382e0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:24.898050 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 605/1181] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:22:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:24.903934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654e9cd-b500-449b-b066-40b91f32ca52 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:25.004135 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654e9cd-b500-449b-b066-40b91f32ca52 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:25.037252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f291d-e57b-4174-a755-592a19ef1702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:25.037932 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f291d-e57b-4174-a755-592a19ef1702 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:25.038218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f291d-e57b-4174-a755-592a19ef1702 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:25.038493 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f291d-e57b-4174-a755-592a19ef1702 None admin] PATH_INFO: `/v3/users/ff73903a55cd4fa4bf2fccec068ddfbd` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:25.051342 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f291d-e57b-4174-a755-592a19ef1702 None admin] RBAC: Authorizing `identity:get_user(user_id=ff73903a55cd4fa4bf2fccec068ddfbd)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:25.052795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f291d-e57b-4174-a755-592a19ef1702 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:25.054509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 577/1182] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:22:24 2022] GET /identity/v3/users/ff73903a55cd4fa4bf2fccec068ddfbd => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:25.060446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b8b45-0a0f-4c35-812a-3566a92382e0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:25.153383 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:25.160503 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b8b45-0a0f-4c35-812a-3566a92382e0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:25.192598 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418708f-6370-4e33-aae3-700c445b1b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:25.193209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418708f-6370-4e33-aae3-700c445b1b17 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:25.193489 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418708f-6370-4e33-aae3-700c445b1b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:25.193784 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418708f-6370-4e33-aae3-700c445b1b17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:25.259483 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418708f-6370-4e33-aae3-700c445b1b17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:25.260931 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418708f-6370-4e33-aae3-700c445b1b17 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:25.357595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:25.365233 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 606/1183] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:22:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:22:25.371350 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f291d-e57b-4174-a755-592a19ef1702 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:25.420391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:25.472150 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f291d-e57b-4174-a755-592a19ef1702 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:25.505407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a60d1e-6719-47ba-8e39-09cd99d2a02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:25.506086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a60d1e-6719-47ba-8e39-09cd99d2a02a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:25.506361 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a60d1e-6719-47ba-8e39-09cd99d2a02a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:25.506664 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a60d1e-6719-47ba-8e39-09cd99d2a02a None admin] PATH_INFO: `/v3/projects/c03cef6958d944219a1b5c53a7c0163d` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:25.507835 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a60d1e-6719-47ba-8e39-09cd99d2a02a None admin] RBAC: Authorizing `identity:get_project(project_id=c03cef6958d944219a1b5c53a7c0163d)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:25.509275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a60d1e-6719-47ba-8e39-09cd99d2a02a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:25.511035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 578/1184] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:22:25 2022] GET /identity/v3/projects/c03cef6958d944219a1b5c53a7c0163d => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:25.517150 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418708f-6370-4e33-aae3-700c445b1b17 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:25.616158 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418708f-6370-4e33-aae3-700c445b1b17 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:25.653224 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6cea4f-5d3d-4802-b984-45237a1b91a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:25.653910 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cea4f-5d3d-4802-b984-45237a1b91a6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:25.654261 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cea4f-5d3d-4802-b984-45237a1b91a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:25.654585 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cea4f-5d3d-4802-b984-45237a1b91a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:25.655412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6cea4f-5d3d-4802-b984-45237a1b91a6 None admin] RBAC: Adding query filter params (role.id=7bdfa43dc1d046f7965ffe52b74bc58f, scope.project.id=c03cef6958d944219a1b5c53a7c0163d, user.id=ff73903a55cd4fa4bf2fccec068ddfbd) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:22:25.655773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6cea4f-5d3d-4802-b984-45237a1b91a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:25.657229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6cea4f-5d3d-4802-b984-45237a1b91a6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:25.662502 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 607/1185] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:22:25 2022] GET /identity/v3/role_assignments?user.id=ff73903a55cd4fa4bf2fccec068ddfbd&scope.project.id=c03cef6958d944219a1b5c53a7c0163d&role.id=7bdfa43dc1d046f7965ffe52b74bc58f => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:27.666429 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390bc52-cbf3-4d02-9398-106115a9681c None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:27.666991 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390bc52-cbf3-4d02-9398-106115a9681c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:27.667423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390bc52-cbf3-4d02-9398-106115a9681c None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:27.669532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 579/1186] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:22:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:22:27.676350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f83a1-5830-484a-a2be-5982f7e83e89 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:27.676785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f83a1-5830-484a-a2be-5982f7e83e89 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:27.676912 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f83a1-5830-484a-a2be-5982f7e83e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:27.694613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c72f83a1-5830-484a-a2be-5982f7e83e89 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:22:27.695790 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c72f83a1-5830-484a-a2be-5982f7e83e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:27.748220 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:27.761896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 608/1187] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:22:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:22:27.784469 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390bc52-cbf3-4d02-9398-106115a9681c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:27.886405 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390bc52-cbf3-4d02-9398-106115a9681c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:27.902727 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:27.923612 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b9ae7-dc7f-4c76-b716-37b6fe02e0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:27.924184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b9ae7-dc7f-4c76-b716-37b6fe02e0ec None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:27.924467 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b9ae7-dc7f-4c76-b716-37b6fe02e0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:27.924767 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b9ae7-dc7f-4c76-b716-37b6fe02e0ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:27.938792 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b9ae7-dc7f-4c76-b716-37b6fe02e0ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:22:27.943721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-8c0b9ae7-dc7f-4c76-b716-37b6fe02e0ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:22:27.943721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:22:27.943721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:22:27.943721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:22:27.943721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:22:27.943721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:22:27.943721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:22:27.943721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:22:27.943721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:22:27.943721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:22:27.943721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:22:27.943721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:22:27.943721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:22:27.943721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:22:27.943721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:22:27.943721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:22:27.944852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:22:27.944852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:22:27.944852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:22:27.944852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:22:27.944852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:22:27.944852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:22:27.944852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:22:27.944852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:22:27.944852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:22:27.944852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:22:27.944852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:22:27.944852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:22:27.944852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:22:27.944852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:22:27.944852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:22:27.944852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:22:27.946088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:22:27.946088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:22:27.946088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:22:27.946088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:22:27.946088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:22:27.946088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:22:27.946088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:22:27.946088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:22:27.946088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:22:27.946088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:22:27.946088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:22:27.946088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:22:27.946088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 580/1188] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:22:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:22:27.951358 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f83a1-5830-484a-a2be-5982f7e83e89 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:28.057259 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f83a1-5830-484a-a2be-5982f7e83e89 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:28.092028 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c63b1d-3883-4d03-b12f-5968c87f9373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:28.092645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c63b1d-3883-4d03-b12f-5968c87f9373 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:28.092926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c63b1d-3883-4d03-b12f-5968c87f9373 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:28.093197 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c63b1d-3883-4d03-b12f-5968c87f9373 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:28.093932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c63b1d-3883-4d03-b12f-5968c87f9373 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:22:28.094242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c63b1d-3883-4d03-b12f-5968c87f9373 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:28.095602 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c63b1d-3883-4d03-b12f-5968c87f9373 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:28.103457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 609/1189] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:22:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:28.109335 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b9ae7-dc7f-4c76-b716-37b6fe02e0ec None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:28.212040 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b9ae7-dc7f-4c76-b716-37b6fe02e0ec None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:28.245432 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b628347-05a8-4e5b-92a1-92cf59fdb83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:28.246095 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b628347-05a8-4e5b-92a1-92cf59fdb83d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:28.246375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b628347-05a8-4e5b-92a1-92cf59fdb83d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:28.246650 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b628347-05a8-4e5b-92a1-92cf59fdb83d None admin] PATH_INFO: `/v3/users/ff73903a55cd4fa4bf2fccec068ddfbd` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:28.247803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b628347-05a8-4e5b-92a1-92cf59fdb83d None admin] RBAC: Authorizing `identity:get_user(user_id=ff73903a55cd4fa4bf2fccec068ddfbd)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:28.249231 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b628347-05a8-4e5b-92a1-92cf59fdb83d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:28.250852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 581/1190] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:22:28 2022] GET /identity/v3/users/ff73903a55cd4fa4bf2fccec068ddfbd => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:28.257273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c63b1d-3883-4d03-b12f-5968c87f9373 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:28.305646 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:28.357758 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c63b1d-3883-4d03-b12f-5968c87f9373 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:28.391605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d32830-04ab-4755-930e-d23994cc5ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:28.392239 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d32830-04ab-4755-930e-d23994cc5ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:28.392521 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d32830-04ab-4755-930e-d23994cc5ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:28.392797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d32830-04ab-4755-930e-d23994cc5ee3 None admin] PATH_INFO: `/v3/projects/c03cef6958d944219a1b5c53a7c0163d` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:28.393970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d32830-04ab-4755-930e-d23994cc5ee3 None admin] RBAC: Authorizing `identity:get_project(project_id=c03cef6958d944219a1b5c53a7c0163d)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:28.395402 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d32830-04ab-4755-930e-d23994cc5ee3 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:28.397067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 610/1191] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:22:28 2022] GET /identity/v3/projects/c03cef6958d944219a1b5c53a7c0163d => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:28.403204 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b628347-05a8-4e5b-92a1-92cf59fdb83d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:28.446755 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:28.502810 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b628347-05a8-4e5b-92a1-92cf59fdb83d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:28.537968 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20548c-5b0c-4f34-b269-6a373f16e559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:28.538635 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20548c-5b0c-4f34-b269-6a373f16e559 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:28.538920 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20548c-5b0c-4f34-b269-6a373f16e559 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:28.539196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20548c-5b0c-4f34-b269-6a373f16e559 None admin] PATH_INFO: `/v3/projects/c03cef6958d944219a1b5c53a7c0163d/users/ff73903a55cd4fa4bf2fccec068ddfbd/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:28.541130 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20548c-5b0c-4f34-b269-6a373f16e559 None admin] RBAC: Authorizing `identity:create_grant(project_id=c03cef6958d944219a1b5c53a7c0163d, user_id=ff73903a55cd4fa4bf2fccec068ddfbd, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:28.542589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20548c-5b0c-4f34-b269-6a373f16e559 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:28.556264 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 582/1192] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:22:28 2022] PUT /identity/v3/projects/c03cef6958d944219a1b5c53a7c0163d/users/ff73903a55cd4fa4bf2fccec068ddfbd/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:22:30.545562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3192bf-8ba7-47ba-8792-c634d9d83a7c None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:30.546256 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3192bf-8ba7-47ba-8792-c634d9d83a7c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:30.546256 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3192bf-8ba7-47ba-8792-c634d9d83a7c None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:30.547499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 611/1193] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:22:30.552563 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a35cdc-c71d-49ab-9c5f-daa2f824bc54 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:30.552997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a35cdc-c71d-49ab-9c5f-daa2f824bc54 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:30.553091 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a35cdc-c71d-49ab-9c5f-daa2f824bc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:30.571222 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-64a35cdc-c71d-49ab-9c5f-daa2f824bc54 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:22:30.572652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-64a35cdc-c71d-49ab-9c5f-daa2f824bc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:30.641294 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 583/1194] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:22:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:22:30.665308 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3192bf-8ba7-47ba-8792-c634d9d83a7c None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:30.773116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3192bf-8ba7-47ba-8792-c634d9d83a7c None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:30.790465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:30.806935 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d2f90-8c70-4606-a344-9961bd0906e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:30.807538 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d2f90-8c70-4606-a344-9961bd0906e4 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:30.807817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d2f90-8c70-4606-a344-9961bd0906e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:30.808088 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d2f90-8c70-4606-a344-9961bd0906e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:30.818403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d2f90-8c70-4606-a344-9961bd0906e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:22:30.823570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-882d2f90-8c70-4606-a344-9961bd0906e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:22:30.823570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:22:30.823570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:22:30.823570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:22:30.823570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:22:30.823570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:22:30.823570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:22:30.823570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:22:30.823570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:22:30.823570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:22:30.823570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:22:30.823570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:22:30.823570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:22:30.823570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:22:30.823570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:22:30.823570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:22:30.824682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:22:30.824682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:22:30.824682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:22:30.824682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:22:30.824682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:22:30.824682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:22:30.824682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:22:30.824682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:22:30.824682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:22:30.824682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:22:30.824682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:22:30.824682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:22:30.824682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:22:30.824682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:22:30.824682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:22:30.824682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:22:30.830570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:22:30.830570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:22:30.830570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:22:30.830570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:22:30.830570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:22:30.830570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:22:30.830570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:22:30.830570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:22:30.830570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:22:30.830570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:22:30.830570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:22:30.830570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:22:30.830570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 612/1195] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:22:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:22:30.831587 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a35cdc-c71d-49ab-9c5f-daa2f824bc54 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:30.861261 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:30.928721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a35cdc-c71d-49ab-9c5f-daa2f824bc54 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:30.963124 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031d949-b982-4c5f-8fa6-601548150482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:30.963739 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031d949-b982-4c5f-8fa6-601548150482 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:30.964018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031d949-b982-4c5f-8fa6-601548150482 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:30.964289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031d949-b982-4c5f-8fa6-601548150482 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:30.964995 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031d949-b982-4c5f-8fa6-601548150482 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:22:30.965294 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031d949-b982-4c5f-8fa6-601548150482 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:30.966695 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031d949-b982-4c5f-8fa6-601548150482 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:30.974497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 584/1196] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:22:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:30.980490 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d2f90-8c70-4606-a344-9961bd0906e4 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:31.080603 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d2f90-8c70-4606-a344-9961bd0906e4 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:31.113827 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a91fb-4251-4c10-b762-81cbb107cd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:31.114502 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a91fb-4251-4c10-b762-81cbb107cd20 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:31.114778 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a91fb-4251-4c10-b762-81cbb107cd20 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:31.115046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a91fb-4251-4c10-b762-81cbb107cd20 None admin] PATH_INFO: `/v3/users/ff73903a55cd4fa4bf2fccec068ddfbd` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:31.116190 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a91fb-4251-4c10-b762-81cbb107cd20 None admin] RBAC: Authorizing `identity:get_user(user_id=ff73903a55cd4fa4bf2fccec068ddfbd)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:31.117713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a91fb-4251-4c10-b762-81cbb107cd20 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:31.119354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 613/1197] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Aug 1 15:22:30 2022] GET /identity/v3/users/ff73903a55cd4fa4bf2fccec068ddfbd => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:31.125132 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031d949-b982-4c5f-8fa6-601548150482 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:31.216222 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:31.223675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031d949-b982-4c5f-8fa6-601548150482 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:31.255685 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbc84e-ba6c-4bfb-82a9-deb8fed2d1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:31.256293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbc84e-ba6c-4bfb-82a9-deb8fed2d1c5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:31.256579 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbc84e-ba6c-4bfb-82a9-deb8fed2d1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:31.256851 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbc84e-ba6c-4bfb-82a9-deb8fed2d1c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:31.322608 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbc84e-ba6c-4bfb-82a9-deb8fed2d1c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:31.324048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbc84e-ba6c-4bfb-82a9-deb8fed2d1c5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:31.411428 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:31.419008 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 585/1198] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:22:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:22:31.424939 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a91fb-4251-4c10-b762-81cbb107cd20 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:31.471677 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:31.525609 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a91fb-4251-4c10-b762-81cbb107cd20 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:31.558273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a97a5-e82d-4664-81dc-f476aa22f2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:31.558914 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a97a5-e82d-4664-81dc-f476aa22f2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:31.559178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a97a5-e82d-4664-81dc-f476aa22f2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:31.559453 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a97a5-e82d-4664-81dc-f476aa22f2d8 None admin] PATH_INFO: `/v3/projects/c03cef6958d944219a1b5c53a7c0163d` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:31.560610 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a97a5-e82d-4664-81dc-f476aa22f2d8 None admin] RBAC: Authorizing `identity:get_project(project_id=c03cef6958d944219a1b5c53a7c0163d)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:31.562067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a97a5-e82d-4664-81dc-f476aa22f2d8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:31.563735 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 614/1199] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Aug 1 15:22:31 2022] GET /identity/v3/projects/c03cef6958d944219a1b5c53a7c0163d => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:31.570127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbc84e-ba6c-4bfb-82a9-deb8fed2d1c5 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:31.676357 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbc84e-ba6c-4bfb-82a9-deb8fed2d1c5 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:31.709935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f926f-d46e-460a-9934-347dd584bd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:31.710547 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f926f-d46e-460a-9934-347dd584bd03 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:31.710845 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f926f-d46e-460a-9934-347dd584bd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:31.711111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f926f-d46e-460a-9934-347dd584bd03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:31.711881 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f926f-d46e-460a-9934-347dd584bd03 None admin] RBAC: Adding query filter params (role.id=7bdfa43dc1d046f7965ffe52b74bc58f, scope.project.id=c03cef6958d944219a1b5c53a7c0163d, user.id=ff73903a55cd4fa4bf2fccec068ddfbd) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:22:31.712187 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f926f-d46e-460a-9934-347dd584bd03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:31.713619 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f926f-d46e-460a-9934-347dd584bd03 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:31.719278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 586/1200] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:22:31 2022] GET /identity/v3/role_assignments?user.id=ff73903a55cd4fa4bf2fccec068ddfbd&scope.project.id=c03cef6958d944219a1b5c53a7c0163d&role.id=7bdfa43dc1d046f7965ffe52b74bc58f => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:22:41.553577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=43 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:41.717114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=71 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:27.921871 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49c4a7-eeab-4fae-9fd8-8d2dc244a2ac None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:27.922508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49c4a7-eeab-4fae-9fd8-8d2dc244a2ac None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:27.922508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49c4a7-eeab-4fae-9fd8-8d2dc244a2ac None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:27.924003 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 615/1201] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:23:27.930388 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2aa23-95dd-4879-b11f-0f29880b2359 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:27.930915 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2aa23-95dd-4879-b11f-0f29880b2359 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:27.931133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2aa23-95dd-4879-b11f-0f29880b2359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:27.958193 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d7e2aa23-95dd-4879-b11f-0f29880b2359 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:27.959636 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d7e2aa23-95dd-4879-b11f-0f29880b2359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:28.053930 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 587/1202] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:23:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:23:28.080335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49c4a7-eeab-4fae-9fd8-8d2dc244a2ac None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:28.217094 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49c4a7-eeab-4fae-9fd8-8d2dc244a2ac None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:28.262258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e37f2b-5823-4c77-ab7a-69231696ca2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:28.263382 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e37f2b-5823-4c77-ab7a-69231696ca2b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:28.263795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e37f2b-5823-4c77-ab7a-69231696ca2b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:28.264145 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e37f2b-5823-4c77-ab7a-69231696ca2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:28.282646 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e37f2b-5823-4c77-ab7a-69231696ca2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:28.284818 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e37f2b-5823-4c77-ab7a-69231696ca2b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:28.306970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-51e37f2b-5823-4c77-ab7a-69231696ca2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:28.306970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:28.306970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:28.306970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:28.306970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:28.306970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:28.306970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:28.306970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:28.306970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:28.306970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:28.306970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:28.306970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:28.306970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:23:28.306970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:23:28.306970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:23:28.308132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:23:28.308132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:28.308132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:28.308132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:28.308132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:28.308132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:28.308132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:28.308132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:28.308132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:28.308132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:28.308132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:28.308132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:28.308132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:28.308132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:28.308132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:28.308132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:28.308132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:28.309225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:23:28.309225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:23:28.309225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:28.309225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:23:28.314381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 616/1203] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:23:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:28.317594 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2aa23-95dd-4879-b11f-0f29880b2359 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:28.452521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2aa23-95dd-4879-b11f-0f29880b2359 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:28.487492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:28.496493 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b4e3e-1709-444e-81a5-21343181e7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:28.497925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b4e3e-1709-444e-81a5-21343181e7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:28.498524 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b4e3e-1709-444e-81a5-21343181e7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:28.498810 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b4e3e-1709-444e-81a5-21343181e7c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:28.499595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b4e3e-1709-444e-81a5-21343181e7c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:28.499912 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b4e3e-1709-444e-81a5-21343181e7c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:28.501359 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b4e3e-1709-444e-81a5-21343181e7c5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:28.522760 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 588/1204] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:23:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:23:28.524799 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e37f2b-5823-4c77-ab7a-69231696ca2b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:28.610486 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:28.674124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e37f2b-5823-4c77-ab7a-69231696ca2b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:28.724039 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327ebaa-318b-4511-beeb-1907cb589f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:28.724998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327ebaa-318b-4511-beeb-1907cb589f8b None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:28.725396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327ebaa-318b-4511-beeb-1907cb589f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:28.725826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327ebaa-318b-4511-beeb-1907cb589f8b None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:28.726636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327ebaa-318b-4511-beeb-1907cb589f8b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:28.728420 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327ebaa-318b-4511-beeb-1907cb589f8b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:28.757089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 617/1205] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Aug 1 15:23:28 2022] POST /identity/v3/users => generated 251 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:23:30.919334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418890b-acf5-49ae-a9d7-c080d1cd908b None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:30.919881 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418890b-acf5-49ae-a9d7-c080d1cd908b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:30.919881 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418890b-acf5-49ae-a9d7-c080d1cd908b None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:30.922811 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 589/1206] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:23:30.926692 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4587d1b-8433-4301-94cb-c3f0174cf148 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:30.927347 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4587d1b-8433-4301-94cb-c3f0174cf148 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:30.927347 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4587d1b-8433-4301-94cb-c3f0174cf148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:30.953455 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c4587d1b-8433-4301-94cb-c3f0174cf148 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:30.954970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c4587d1b-8433-4301-94cb-c3f0174cf148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:31.054785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 618/1207] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:23:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:23:31.078003 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418890b-acf5-49ae-a9d7-c080d1cd908b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:31.199710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418890b-acf5-49ae-a9d7-c080d1cd908b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:31.242049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d70a02-6cee-4867-87f6-afd6fc6e033e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:31.242805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d70a02-6cee-4867-87f6-afd6fc6e033e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:31.243132 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d70a02-6cee-4867-87f6-afd6fc6e033e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:31.243461 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d70a02-6cee-4867-87f6-afd6fc6e033e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:31.257971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d70a02-6cee-4867-87f6-afd6fc6e033e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:23:31.263841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-33d70a02-6cee-4867-87f6-afd6fc6e033e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:23:31.263841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:31.263841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:31.263841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:31.263841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:31.263841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:31.263841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:31.263841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:31.263841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:31.263841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:31.263841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:31.263841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:31.263841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:23:31.263841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:23:31.263841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:23:31.263841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:23:31.264837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:23:31.264837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:23:31.264837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:23:31.264837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:23:31.264837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:31.264837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:31.264837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:31.264837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:31.264837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:31.264837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:31.264837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:31.264837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:31.264837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:31.264837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:31.264837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:31.264837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:31.271017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:31.271017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:31.271017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:31.271017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:31.271017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:23:31.271017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:23:31.271017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:23:31.271017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:23:31.271017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:23:31.271017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:23:31.271017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:23:31.271017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:23:31.271017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 590/1208] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:23:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:31.271912 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4587d1b-8433-4301-94cb-c3f0174cf148 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:31.324477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:31.396279 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4587d1b-8433-4301-94cb-c3f0174cf148 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:31.436627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d0031-a1ad-4c68-a5af-cf8a6e1fdada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:31.437261 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d0031-a1ad-4c68-a5af-cf8a6e1fdada None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:31.437654 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d0031-a1ad-4c68-a5af-cf8a6e1fdada None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:31.437969 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d0031-a1ad-4c68-a5af-cf8a6e1fdada None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:31.438816 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d0031-a1ad-4c68-a5af-cf8a6e1fdada None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:31.439142 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d0031-a1ad-4c68-a5af-cf8a6e1fdada None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:31.440651 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d0031-a1ad-4c68-a5af-cf8a6e1fdada None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:31.455741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 619/1209] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:23:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:23:31.456915 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d70a02-6cee-4867-87f6-afd6fc6e033e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:31.515138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:31.574432 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d70a02-6cee-4867-87f6-afd6fc6e033e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:31.612797 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c3293-0b87-4a66-b2fe-3ec8f8804464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:31.613451 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c3293-0b87-4a66-b2fe-3ec8f8804464 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:31.613736 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c3293-0b87-4a66-b2fe-3ec8f8804464 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:31.614049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c3293-0b87-4a66-b2fe-3ec8f8804464 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:31.632876 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c3293-0b87-4a66-b2fe-3ec8f8804464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:31.634568 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c3293-0b87-4a66-b2fe-3ec8f8804464 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:31.652936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-393c3293-0b87-4a66-b2fe-3ec8f8804464 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:31.652936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:31.652936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:31.652936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:31.652936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:31.652936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:31.652936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:31.652936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:31.652936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:31.652936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:31.652936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:31.652936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:31.652936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:23:31.652936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:23:31.652936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:23:31.653985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:23:31.653985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:31.653985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:31.653985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:31.653985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:31.653985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:31.653985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:31.653985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:31.653985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:31.653985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:31.653985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:31.653985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:31.653985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:31.653985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:31.653985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:31.653985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:31.653985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:31.654937 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:23:31.654937 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:23:31.654937 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:31.654937 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:23:31.654937 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 591/1210] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:23:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:23:31.660067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d0031-a1ad-4c68-a5af-cf8a6e1fdada None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:31.766178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:31.769808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d0031-a1ad-4c68-a5af-cf8a6e1fdada None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:31.813814 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07237fe2-232c-4ae8-8c25-2fae2db25466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:31.814678 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07237fe2-232c-4ae8-8c25-2fae2db25466 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:31.816012 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07237fe2-232c-4ae8-8c25-2fae2db25466 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:31.816012 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07237fe2-232c-4ae8-8c25-2fae2db25466 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:31.816710 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07237fe2-232c-4ae8-8c25-2fae2db25466 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:31.817025 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07237fe2-232c-4ae8-8c25-2fae2db25466 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:31.818747 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07237fe2-232c-4ae8-8c25-2fae2db25466 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:31.843128 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 620/1211] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:23:31.844305 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c3293-0b87-4a66-b2fe-3ec8f8804464 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:31.982000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c3293-0b87-4a66-b2fe-3ec8f8804464 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:31.983695 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:32.020779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12308e27-74ca-4bab-bd25-c78e36b43009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:32.021356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12308e27-74ca-4bab-bd25-c78e36b43009 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:32.021637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12308e27-74ca-4bab-bd25-c78e36b43009 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:32.021966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12308e27-74ca-4bab-bd25-c78e36b43009 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:32.031980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12308e27-74ca-4bab-bd25-c78e36b43009 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:32.033516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12308e27-74ca-4bab-bd25-c78e36b43009 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:32.045730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-12308e27-74ca-4bab-bd25-c78e36b43009 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:23:32.045730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:32.045730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:32.045730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:32.045730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:32.045730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:32.045730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:32.045730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:32.045730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:32.045730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:32.045730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:32.045730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:32.045730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:23:32.045730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:23:32.045730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:23:32.045730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:23:32.047115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:32.047115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:32.047115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:23:32.047115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:23:32.047115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:23:32.047115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:23:32.047115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:32.047115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:32.047115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:32.047115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:32.047115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:32.047115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:32.047115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:32.047115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:32.047115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:32.047115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:32.047115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:32.047115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:32.048133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:32.048133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:32.048133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:23:32.048133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:23:32.048133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:23:32.048133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:23:32.048133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:23:32.048133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:23:32.048133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:23:32.048133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:23:32.048133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 592/1212] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:23:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:32.052905 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07237fe2-232c-4ae8-8c25-2fae2db25466 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:32.179811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07237fe2-232c-4ae8-8c25-2fae2db25466 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:32.220746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4abe25-6573-4277-a516-62915f275e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:32.221382 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4abe25-6573-4277-a516-62915f275e17 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:32.221708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4abe25-6573-4277-a516-62915f275e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:32.221930 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4abe25-6573-4277-a516-62915f275e17 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:32.222880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4abe25-6573-4277-a516-62915f275e17 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:32.223186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4abe25-6573-4277-a516-62915f275e17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:32.224688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4abe25-6573-4277-a516-62915f275e17 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:32.248217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 621/1213] 10.222.0.83 () {56 vars in 1143 bytes} [Mon Aug 1 15:23:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:32.258844 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12308e27-74ca-4bab-bd25-c78e36b43009 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:32.382353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12308e27-74ca-4bab-bd25-c78e36b43009 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:32.428216 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4cd39-2f23-4504-8c58-cf08c456096a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:32.429474 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4cd39-2f23-4504-8c58-cf08c456096a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:32.429800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4cd39-2f23-4504-8c58-cf08c456096a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:32.430167 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4cd39-2f23-4504-8c58-cf08c456096a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:32.471184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:32.522860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4cd39-2f23-4504-8c58-cf08c456096a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:32.524992 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4cd39-2f23-4504-8c58-cf08c456096a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:32.660037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 593/1214] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:23:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:23:32.668855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4abe25-6573-4277-a516-62915f275e17 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:32.694404 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:32.803262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4abe25-6573-4277-a516-62915f275e17 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:32.855420 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01a9f4-1e47-449a-a544-4e54c487da00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:32.856124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01a9f4-1e47-449a-a544-4e54c487da00 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:32.856401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01a9f4-1e47-449a-a544-4e54c487da00 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:32.856668 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01a9f4-1e47-449a-a544-4e54c487da00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:32.869751 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01a9f4-1e47-449a-a544-4e54c487da00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:32.871256 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01a9f4-1e47-449a-a544-4e54c487da00 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:32.887284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-5b01a9f4-1e47-449a-a544-4e54c487da00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:32.887284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:32.887284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:32.887284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:32.887284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:32.887284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:32.887284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:32.887284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:32.887284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:32.887284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:32.887284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:32.887284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:32.887284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:23:32.887284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:23:32.887284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:23:32.888476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:23:32.888476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:32.888476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:32.888476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:32.888476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:32.888476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:32.888476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:32.888476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:32.888476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:32.888476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:32.888476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:32.888476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:32.888476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:32.888476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:32.888476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:32.888476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:32.888476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:32.889660 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:23:32.889660 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:23:32.889660 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:32.889660 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:23:32.893098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 622/1215] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:23:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:32.894639 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4cd39-2f23-4504-8c58-cf08c456096a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:32.966360 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:33.007254 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4cd39-2f23-4504-8c58-cf08c456096a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:33.044798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4023d80-24e2-4d33-8dc8-193f945ae918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:33.045526 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4023d80-24e2-4d33-8dc8-193f945ae918 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:33.045717 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4023d80-24e2-4d33-8dc8-193f945ae918 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:33.046067 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4023d80-24e2-4d33-8dc8-193f945ae918 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:33.046745 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4023d80-24e2-4d33-8dc8-193f945ae918 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:33.047021 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4023d80-24e2-4d33-8dc8-193f945ae918 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:33.049925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4023d80-24e2-4d33-8dc8-193f945ae918 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:33.065654 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 594/1216] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:33.067117 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01a9f4-1e47-449a-a544-4e54c487da00 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:33.150614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:33.200714 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01a9f4-1e47-449a-a544-4e54c487da00 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:33.249345 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6629936-9db6-4708-8e01-72ae2b7d953d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:33.249941 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6629936-9db6-4708-8e01-72ae2b7d953d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:33.250225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6629936-9db6-4708-8e01-72ae2b7d953d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:33.250493 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6629936-9db6-4708-8e01-72ae2b7d953d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:33.259950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6629936-9db6-4708-8e01-72ae2b7d953d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:33.261494 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6629936-9db6-4708-8e01-72ae2b7d953d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:33.274394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-b6629936-9db6-4708-8e01-72ae2b7d953d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:23:33.274394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:33.274394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:33.274394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:33.274394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:33.274394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:33.274394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:33.274394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:33.274394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:33.274394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:33.274394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:33.274394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:33.274394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:23:33.274394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:23:33.274394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:23:33.275406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:23:33.275406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:33.275406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:33.275406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:33.275406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:33.275406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:33.275406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:33.275406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:33.275406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:33.275406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:33.275406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:33.275406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:33.275406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:33.275406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:33.275406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:33.275406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:33.275406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:33.276441 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:23:33.276441 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:23:33.276441 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:23:33.276441 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:23:33.276441 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:23:33.276441 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:23:33.276441 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:23:33.276441 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:23:33.276441 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 623/1217] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:23:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:33.281714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4023d80-24e2-4d33-8dc8-193f945ae918 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:33.402131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4023d80-24e2-4d33-8dc8-193f945ae918 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:33.429452 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:33.443710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdecbe9a-f0b0-4f37-a02a-dca6920b7659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:33.444870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdecbe9a-f0b0-4f37-a02a-dca6920b7659 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:33.445591 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdecbe9a-f0b0-4f37-a02a-dca6920b7659 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:33.446052 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdecbe9a-f0b0-4f37-a02a-dca6920b7659 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:33.447165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdecbe9a-f0b0-4f37-a02a-dca6920b7659 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:33.447962 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdecbe9a-f0b0-4f37-a02a-dca6920b7659 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:33.450500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdecbe9a-f0b0-4f37-a02a-dca6920b7659 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:33.470035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 595/1218] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:23:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:33.476713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6629936-9db6-4708-8e01-72ae2b7d953d None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:33.625430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6629936-9db6-4708-8e01-72ae2b7d953d None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:33.657443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:33.664837 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc2ea7-bfc5-4160-a685-51bad7bd3885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:33.665496 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc2ea7-bfc5-4160-a685-51bad7bd3885 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:33.665774 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc2ea7-bfc5-4160-a685-51bad7bd3885 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:33.666123 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc2ea7-bfc5-4160-a685-51bad7bd3885 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:33.666888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc2ea7-bfc5-4160-a685-51bad7bd3885 None admin] RBAC: Adding query filter params (role.id=091d39715077464daa4cf890db55e6b9, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=ef57a9394b114eb3ab499133b5d40781) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:33.667300 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc2ea7-bfc5-4160-a685-51bad7bd3885 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:33.668789 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc2ea7-bfc5-4160-a685-51bad7bd3885 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:33.686166 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 624/1219] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:23:33 2022] GET /identity/v3/role_assignments?user.id=ef57a9394b114eb3ab499133b5d40781&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=091d39715077464daa4cf890db55e6b9 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:35.868181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee883b-c8da-4273-bf54-6c81ee6530ff None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:35.868803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee883b-c8da-4273-bf54-6c81ee6530ff None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:35.868803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee883b-c8da-4273-bf54-6c81ee6530ff None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:35.870420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 596/1220] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:23:35.875945 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bf2c8-a101-41bf-9b34-d233ed165fc1 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:35.876509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bf2c8-a101-41bf-9b34-d233ed165fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:35.876509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bf2c8-a101-41bf-9b34-d233ed165fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:35.898368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-b36bf2c8-a101-41bf-9b34-d233ed165fc1 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:35.900646 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b36bf2c8-a101-41bf-9b34-d233ed165fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:35.990473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 625/1221] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:23:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:23:36.014376 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee883b-c8da-4273-bf54-6c81ee6530ff None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:36.126853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee883b-c8da-4273-bf54-6c81ee6530ff None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:36.166432 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd638a5c-7797-4391-97d0-91eda7a16779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:36.167088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd638a5c-7797-4391-97d0-91eda7a16779 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:36.167364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd638a5c-7797-4391-97d0-91eda7a16779 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:36.167633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd638a5c-7797-4391-97d0-91eda7a16779 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:36.179818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd638a5c-7797-4391-97d0-91eda7a16779 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:23:36.184749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-dd638a5c-7797-4391-97d0-91eda7a16779 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:23:36.184749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:36.184749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:36.184749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:36.184749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:36.184749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:36.184749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:36.184749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:36.184749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:36.184749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:36.184749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:36.184749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:36.184749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:23:36.184749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:23:36.184749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:23:36.184749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:23:36.185869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:23:36.185869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:23:36.185869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:23:36.185869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:23:36.185869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:36.185869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:36.185869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:36.185869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:36.185869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:36.185869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:36.185869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:36.185869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:36.185869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:36.185869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:36.185869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:36.185869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:36.186879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:36.186879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:36.186879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:36.186879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:36.186879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:23:36.186879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:23:36.186879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:23:36.186879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:23:36.186879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:23:36.186879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:23:36.186879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:23:36.186879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:23:36.186879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 597/1222] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:23:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:36.191988 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36bf2c8-a101-41bf-9b34-d233ed165fc1 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:36.300992 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36bf2c8-a101-41bf-9b34-d233ed165fc1 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:36.332048 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:36.345586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41347fe-6ba1-4c97-a07c-a15933211fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:36.346339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41347fe-6ba1-4c97-a07c-a15933211fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:36.346670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41347fe-6ba1-4c97-a07c-a15933211fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:36.347025 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41347fe-6ba1-4c97-a07c-a15933211fc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:36.347912 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41347fe-6ba1-4c97-a07c-a15933211fc4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:36.348251 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41347fe-6ba1-4c97-a07c-a15933211fc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:36.349753 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41347fe-6ba1-4c97-a07c-a15933211fc4 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:36.359232 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 626/1223] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:23:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:36.365978 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd638a5c-7797-4391-97d0-91eda7a16779 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:36.417653 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:36.476913 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd638a5c-7797-4391-97d0-91eda7a16779 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:36.516810 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a92909-d8aa-4eb5-814f-7d092cc08268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:36.517578 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a92909-d8aa-4eb5-814f-7d092cc08268 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:36.517857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a92909-d8aa-4eb5-814f-7d092cc08268 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:36.518223 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a92909-d8aa-4eb5-814f-7d092cc08268 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:36.532703 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a92909-d8aa-4eb5-814f-7d092cc08268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:36.534445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a92909-d8aa-4eb5-814f-7d092cc08268 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:36.550529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-85a92909-d8aa-4eb5-814f-7d092cc08268 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:36.550529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:36.550529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:36.550529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:36.550529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:36.550529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:36.550529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:36.550529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:36.550529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:36.550529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:36.550529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:36.550529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:36.550529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:23:36.550529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:23:36.550529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:23:36.557101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:23:36.557101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:36.557101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:36.557101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:36.557101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:36.557101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:36.557101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:36.557101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:36.557101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:36.557101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:36.557101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:36.557101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:36.557101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:36.557101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:36.557101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:36.557101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:36.557101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:36.558307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:23:36.558307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:23:36.558307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:36.558307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:23:36.558307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 598/1224] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:23:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:36.559415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41347fe-6ba1-4c97-a07c-a15933211fc4 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:36.671055 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41347fe-6ba1-4c97-a07c-a15933211fc4 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:36.707053 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea2525-5531-45a8-95b5-6d7056c9becf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:36.707835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea2525-5531-45a8-95b5-6d7056c9becf None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:36.708275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea2525-5531-45a8-95b5-6d7056c9becf None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:36.708564 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea2525-5531-45a8-95b5-6d7056c9becf None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:36.709540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea2525-5531-45a8-95b5-6d7056c9becf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:36.710022 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea2525-5531-45a8-95b5-6d7056c9becf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:36.711998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea2525-5531-45a8-95b5-6d7056c9becf None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:36.726112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 627/1225] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:36.732620 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a92909-d8aa-4eb5-814f-7d092cc08268 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:36.842664 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:36.849667 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a92909-d8aa-4eb5-814f-7d092cc08268 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:36.887356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fbf197-b9e0-4154-8756-1df422aee713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:36.888057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbf197-b9e0-4154-8756-1df422aee713 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:36.888298 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbf197-b9e0-4154-8756-1df422aee713 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:36.888558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbf197-b9e0-4154-8756-1df422aee713 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:36.897005 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fbf197-b9e0-4154-8756-1df422aee713 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:36.898544 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fbf197-b9e0-4154-8756-1df422aee713 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:36.910344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-f3fbf197-b9e0-4154-8756-1df422aee713 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:23:36.910344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:36.910344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:36.910344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:36.910344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:36.910344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:36.910344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:36.910344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:36.910344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:36.910344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:36.910344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:36.910344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:36.910344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:23:36.910344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:23:36.910344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:23:36.910344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:23:36.911860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:36.911860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:36.911860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:23:36.911860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:23:36.911860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:23:36.911860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:23:36.911860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:36.911860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:36.911860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:36.911860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:36.911860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:36.911860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:36.911860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:36.911860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:36.911860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:36.911860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:36.911860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:36.911860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:36.913155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:36.913155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:36.913155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:23:36.913155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:23:36.913155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:23:36.913155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:23:36.913155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:23:36.913155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:23:36.913155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:23:36.913155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:23:36.913155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 599/1226] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:23:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:36.917634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea2525-5531-45a8-95b5-6d7056c9becf None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:36.972613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:37.038753 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea2525-5531-45a8-95b5-6d7056c9becf None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:37.083125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f2dac-127d-4f2e-a97a-b8b921ff2e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:37.084028 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f2dac-127d-4f2e-a97a-b8b921ff2e73 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:37.084434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f2dac-127d-4f2e-a97a-b8b921ff2e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:37.084743 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f2dac-127d-4f2e-a97a-b8b921ff2e73 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:37.085871 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f2dac-127d-4f2e-a97a-b8b921ff2e73 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:37.086496 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f2dac-127d-4f2e-a97a-b8b921ff2e73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:37.088377 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f2dac-127d-4f2e-a97a-b8b921ff2e73 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:37.113987 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 628/1227] 10.222.0.83 () {56 vars in 1143 bytes} [Mon Aug 1 15:23:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:23:37.120565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fbf197-b9e0-4154-8756-1df422aee713 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:37.247337 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fbf197-b9e0-4154-8756-1df422aee713 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:37.289329 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde78fe-223c-4f80-99c8-4e05057c62ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:37.290283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde78fe-223c-4f80-99c8-4e05057c62ff None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:37.290614 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde78fe-223c-4f80-99c8-4e05057c62ff None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:37.290983 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde78fe-223c-4f80-99c8-4e05057c62ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:37.305698 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde78fe-223c-4f80-99c8-4e05057c62ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:37.307472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde78fe-223c-4f80-99c8-4e05057c62ff None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:37.327013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-ffde78fe-223c-4f80-99c8-4e05057c62ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:37.327013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:37.327013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:37.327013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:37.327013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:37.327013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:37.327013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:37.327013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:37.327013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:37.327013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:37.327013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:37.327013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:37.327013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:23:37.327013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:23:37.327013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:23:37.328148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:23:37.328148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:37.328148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:37.328148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:37.328148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:37.328148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:37.328148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:37.328148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:37.328148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:37.328148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:37.328148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:37.328148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:37.328148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:37.328148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:37.328148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:37.328148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:37.328148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:37.329208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:23:37.329208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:23:37.329208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:37.329208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:23:37.331137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 600/1228] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:23:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:37.337871 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f2dac-127d-4f2e-a97a-b8b921ff2e73 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:37.463953 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f2dac-127d-4f2e-a97a-b8b921ff2e73 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:37.466591 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:37.503550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081adbe-41a9-401f-a745-3cf5981d21e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:37.504093 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081adbe-41a9-401f-a745-3cf5981d21e7 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:37.504381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081adbe-41a9-401f-a745-3cf5981d21e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:37.504643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081adbe-41a9-401f-a745-3cf5981d21e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:37.505379 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081adbe-41a9-401f-a745-3cf5981d21e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:37.505681 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081adbe-41a9-401f-a745-3cf5981d21e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:37.507172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081adbe-41a9-401f-a745-3cf5981d21e7 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:37.532441 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 629/1229] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:37.534041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde78fe-223c-4f80-99c8-4e05057c62ff None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:37.545276 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:37.664551 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde78fe-223c-4f80-99c8-4e05057c62ff None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:37.715995 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4c985-7a3d-41c5-a1c3-5fac6739a681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:37.716685 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4c985-7a3d-41c5-a1c3-5fac6739a681 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:37.716933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4c985-7a3d-41c5-a1c3-5fac6739a681 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:37.717235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4c985-7a3d-41c5-a1c3-5fac6739a681 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:37.724249 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4c985-7a3d-41c5-a1c3-5fac6739a681 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:37.725766 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4c985-7a3d-41c5-a1c3-5fac6739a681 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:37.738308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-64d4c985-7a3d-41c5-a1c3-5fac6739a681 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:23:37.738308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:37.738308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:37.738308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:37.738308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:37.738308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:37.738308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:37.738308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:37.738308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:37.738308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:37.738308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:37.738308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:37.738308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:23:37.738308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:23:37.738308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:23:37.739442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:23:37.739442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:37.739442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:37.739442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:37.739442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:37.739442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:37.739442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:37.739442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:37.739442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:37.739442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:37.739442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:37.739442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:37.739442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:37.739442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:37.739442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:37.739442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:37.739442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:37.740391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:23:37.740391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:23:37.740391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:23:37.740391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:23:37.740391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:23:37.740391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:23:37.740391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:23:37.740391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:23:37.740391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 601/1230] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:23:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:37.745655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081adbe-41a9-401f-a745-3cf5981d21e7 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:37.861377 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081adbe-41a9-401f-a745-3cf5981d21e7 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:37.896900 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533d32b-54a2-4886-b480-287ad49b9ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:37.898268 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533d32b-54a2-4886-b480-287ad49b9ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:37.898629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533d32b-54a2-4886-b480-287ad49b9ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:37.898928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533d32b-54a2-4886-b480-287ad49b9ce8 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:37.899810 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533d32b-54a2-4886-b480-287ad49b9ce8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:37.900185 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533d32b-54a2-4886-b480-287ad49b9ce8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:37.902037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533d32b-54a2-4886-b480-287ad49b9ce8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:37.917376 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 630/1231] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:23:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:23:37.923719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4c985-7a3d-41c5-a1c3-5fac6739a681 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:37.993013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:38.030772 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4c985-7a3d-41c5-a1c3-5fac6739a681 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:38.069485 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd8ce8-84e6-4d6f-ab59-4c532f879b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:38.070360 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd8ce8-84e6-4d6f-ab59-4c532f879b42 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:38.070786 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd8ce8-84e6-4d6f-ab59-4c532f879b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:38.071255 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd8ce8-84e6-4d6f-ab59-4c532f879b42 None admin] PATH_INFO: `/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/ef57a9394b114eb3ab499133b5d40781/roles/091d39715077464daa4cf890db55e6b9` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:38.088802 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd8ce8-84e6-4d6f-ab59-4c532f879b42 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f4e27cc50bd4f9f8ced4cc544ab3064, user_id=ef57a9394b114eb3ab499133b5d40781, role_id=091d39715077464daa4cf890db55e6b9)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:38.090423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd8ce8-84e6-4d6f-ab59-4c532f879b42 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:38.103971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 602/1232] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:23:37 2022] PUT /identity/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/ef57a9394b114eb3ab499133b5d40781/roles/091d39715077464daa4cf890db55e6b9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:23:40.482134 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5837fa7-044b-4120-83d2-d812bb6a94c8 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.482705 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5837fa7-044b-4120-83d2-d812bb6a94c8 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.482705 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5837fa7-044b-4120-83d2-d812bb6a94c8 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.484025 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 631/1233] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:23:40.490260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4d673-4d9f-448f-9cd1-a7d83690e7e9 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.490746 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4d673-4d9f-448f-9cd1-a7d83690e7e9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.490926 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4d673-4d9f-448f-9cd1-a7d83690e7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.512686 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-77e4d673-4d9f-448f-9cd1-a7d83690e7e9 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:40.513962 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-77e4d673-4d9f-448f-9cd1-a7d83690e7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:40.595063 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 603/1234] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:23:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:23:40.619824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5837fa7-044b-4120-83d2-d812bb6a94c8 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.648744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:40.732523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5837fa7-044b-4120-83d2-d812bb6a94c8 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.776193 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b83caf-189b-4bbf-af1e-50a64999c880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.777135 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b83caf-189b-4bbf-af1e-50a64999c880 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.777515 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b83caf-189b-4bbf-af1e-50a64999c880 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.777787 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b83caf-189b-4bbf-af1e-50a64999c880 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.789313 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b83caf-189b-4bbf-af1e-50a64999c880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:23:40.795401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-a5b83caf-189b-4bbf-af1e-50a64999c880 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:23:40.795401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:40.795401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:40.795401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:40.795401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:40.795401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:40.795401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:40.795401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:40.795401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:40.795401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:40.795401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:40.795401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:40.795401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:23:40.795401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:23:40.795401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:23:40.795401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:23:40.796693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:23:40.796693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:23:40.796693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:23:40.796693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:23:40.796693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:40.796693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:40.796693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:40.796693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:40.796693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:40.796693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:40.796693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:40.796693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:40.796693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:40.796693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:40.796693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:40.796693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:40.797645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:40.797645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:40.797645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:40.797645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:40.797645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:23:40.797645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:23:40.797645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:23:40.797645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:23:40.797645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:23:40.797645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:23:40.797645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:23:40.797645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:23:40.797645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 632/1235] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Aug 1 15:23:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:40.803408 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4d673-4d9f-448f-9cd1-a7d83690e7e9 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.860661 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:40.933035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4d673-4d9f-448f-9cd1-a7d83690e7e9 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.973807 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb9033-feb9-4439-9222-28437be124e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.974592 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb9033-feb9-4439-9222-28437be124e3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.974973 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb9033-feb9-4439-9222-28437be124e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.975413 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb9033-feb9-4439-9222-28437be124e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.976217 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb9033-feb9-4439-9222-28437be124e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:40.976533 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb9033-feb9-4439-9222-28437be124e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.978369 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb9033-feb9-4439-9222-28437be124e3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.993536 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 604/1236] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Aug 1 15:23:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:40.995027 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b83caf-189b-4bbf-af1e-50a64999c880 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:41.091651 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:41.117552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b83caf-189b-4bbf-af1e-50a64999c880 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:41.156130 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fcfa6-1a6e-4a8e-b54c-a3d16ecf6607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:41.156775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fcfa6-1a6e-4a8e-b54c-a3d16ecf6607 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.157051 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fcfa6-1a6e-4a8e-b54c-a3d16ecf6607 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.157320 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fcfa6-1a6e-4a8e-b54c-a3d16ecf6607 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.170610 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7fcfa6-1a6e-4a8e-b54c-a3d16ecf6607 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:41.172238 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7fcfa6-1a6e-4a8e-b54c-a3d16ecf6607 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:41.188989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-6e7fcfa6-1a6e-4a8e-b54c-a3d16ecf6607 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:41.188989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:41.188989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:41.188989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:41.188989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:41.188989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:41.188989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:41.188989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:41.188989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:41.188989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:41.188989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:41.188989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:41.188989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:23:41.188989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:23:41.188989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:23:41.190319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:23:41.190319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:41.190319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:41.190319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:41.190319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:41.190319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:41.190319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:41.190319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:41.190319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:41.190319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:41.190319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:41.190319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:41.190319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:41.190319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:41.190319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:41.190319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:41.190319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:41.191263 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:23:41.191263 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:23:41.191263 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:41.191263 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:23:41.191263 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 633/1237] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:23:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:41.197735 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb9033-feb9-4439-9222-28437be124e3 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:41.311427 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb9033-feb9-4439-9222-28437be124e3 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:41.321402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:41.349420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb732d-e862-40bb-9fc9-8ea2ef3423f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:41.350193 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb732d-e862-40bb-9fc9-8ea2ef3423f2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.350476 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb732d-e862-40bb-9fc9-8ea2ef3423f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.350751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb732d-e862-40bb-9fc9-8ea2ef3423f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.351441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb732d-e862-40bb-9fc9-8ea2ef3423f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:41.351749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb732d-e862-40bb-9fc9-8ea2ef3423f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:41.353334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb732d-e862-40bb-9fc9-8ea2ef3423f2 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:41.369266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 605/1238] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:41.374869 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fcfa6-1a6e-4a8e-b54c-a3d16ecf6607 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:41.488979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fcfa6-1a6e-4a8e-b54c-a3d16ecf6607 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:41.538671 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455647ab-3979-40de-bd2c-18f3ae2b66e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:41.539320 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455647ab-3979-40de-bd2c-18f3ae2b66e9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.539691 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455647ab-3979-40de-bd2c-18f3ae2b66e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.539994 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455647ab-3979-40de-bd2c-18f3ae2b66e9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.546055 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:41.554168 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455647ab-3979-40de-bd2c-18f3ae2b66e9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:41.556019 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455647ab-3979-40de-bd2c-18f3ae2b66e9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:41.568636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-455647ab-3979-40de-bd2c-18f3ae2b66e9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:23:41.568636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:41.568636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:41.568636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:41.568636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:41.568636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:41.568636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:41.568636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:41.568636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:41.568636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:41.568636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:41.568636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:41.568636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:23:41.568636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:23:41.568636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:23:41.568636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:23:41.569985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:41.569985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:41.569985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:23:41.569985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:23:41.569985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:23:41.569985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:23:41.569985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:41.569985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:41.569985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:41.569985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:41.569985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:41.569985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:41.569985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:41.569985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:41.569985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:41.569985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:41.569985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:41.569985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:41.571184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:41.571184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:41.571184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:23:41.571184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:23:41.571184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:23:41.571184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:23:41.571184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:23:41.571184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:23:41.571184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:23:41.571184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:23:41.571184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 634/1239] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:23:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:41.576360 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb732d-e862-40bb-9fc9-8ea2ef3423f2 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:41.690649 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb732d-e862-40bb-9fc9-8ea2ef3423f2 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:41.729078 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4ce90-bd02-4c97-b736-df459e38429d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:41.730050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4ce90-bd02-4c97-b736-df459e38429d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.730180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4ce90-bd02-4c97-b736-df459e38429d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.730412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4ce90-bd02-4c97-b736-df459e38429d None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.731594 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4ce90-bd02-4c97-b736-df459e38429d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:41.732049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4ce90-bd02-4c97-b736-df459e38429d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:41.733754 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4ce90-bd02-4c97-b736-df459e38429d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:41.754050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 606/1240] 10.222.0.83 () {56 vars in 1143 bytes} [Mon Aug 1 15:23:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:23:41.760356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455647ab-3979-40de-bd2c-18f3ae2b66e9 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:41.878238 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455647ab-3979-40de-bd2c-18f3ae2b66e9 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:41.919162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46143d0-773e-4dd2-816d-de8e58823a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:41.919917 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46143d0-773e-4dd2-816d-de8e58823a35 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.920235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46143d0-773e-4dd2-816d-de8e58823a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.920539 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46143d0-773e-4dd2-816d-de8e58823a35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.990796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:41.998300 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46143d0-773e-4dd2-816d-de8e58823a35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:41.999809 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46143d0-773e-4dd2-816d-de8e58823a35 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:42.122382 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 635/1241] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:23:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:23:42.128718 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4ce90-bd02-4c97-b736-df459e38429d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:42.161733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:42.250173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4ce90-bd02-4c97-b736-df459e38429d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:42.296130 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54bfea-8cf7-4f3e-915d-b321684adca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:42.297208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54bfea-8cf7-4f3e-915d-b321684adca5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:42.297581 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54bfea-8cf7-4f3e-915d-b321684adca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:42.298066 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54bfea-8cf7-4f3e-915d-b321684adca5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:42.312658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54bfea-8cf7-4f3e-915d-b321684adca5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:42.314525 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54bfea-8cf7-4f3e-915d-b321684adca5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:42.336472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-5c54bfea-8cf7-4f3e-915d-b321684adca5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:42.336472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:42.336472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:42.336472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:42.336472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:42.336472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:42.336472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:42.336472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:42.336472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:42.336472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:42.336472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:42.336472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:42.336472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:23:42.336472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:23:42.336472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:23:42.338504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:23:42.338504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:42.338504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:42.338504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:42.338504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:42.338504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:42.338504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:42.338504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:42.338504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:42.338504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:42.338504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:42.338504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:42.338504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:42.338504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:42.338504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:42.338504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:42.338504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:42.340258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:23:42.340258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:23:42.340258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:42.340258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:23:42.340258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 607/1242] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:23:42.344445 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46143d0-773e-4dd2-816d-de8e58823a35 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:42.455285 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:42.463437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46143d0-773e-4dd2-816d-de8e58823a35 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:42.504804 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b6f75-3604-4120-ad7c-cb3f6f3c650d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:42.505689 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b6f75-3604-4120-ad7c-cb3f6f3c650d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:42.506056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b6f75-3604-4120-ad7c-cb3f6f3c650d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:42.506958 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b6f75-3604-4120-ad7c-cb3f6f3c650d None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:42.508308 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b6f75-3604-4120-ad7c-cb3f6f3c650d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:42.508892 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b6f75-3604-4120-ad7c-cb3f6f3c650d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:42.511018 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b6f75-3604-4120-ad7c-cb3f6f3c650d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:42.531217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 636/1243] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:23:42.539218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54bfea-8cf7-4f3e-915d-b321684adca5 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:42.622730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:42.646761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54bfea-8cf7-4f3e-915d-b321684adca5 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:42.684002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9bff46-2cfc-4a46-bf70-ed3cc5933320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:42.684660 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bff46-2cfc-4a46-bf70-ed3cc5933320 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:42.684945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bff46-2cfc-4a46-bf70-ed3cc5933320 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:42.685221 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bff46-2cfc-4a46-bf70-ed3cc5933320 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:42.692568 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9bff46-2cfc-4a46-bf70-ed3cc5933320 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:42.694139 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9bff46-2cfc-4a46-bf70-ed3cc5933320 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:42.707555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-be9bff46-2cfc-4a46-bf70-ed3cc5933320 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:23:42.707555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:42.707555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:42.707555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:42.707555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:42.707555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:42.707555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:42.707555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:42.707555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:42.707555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:42.707555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:42.707555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:42.707555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:23:42.707555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:23:42.707555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:23:42.708911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:23:42.708911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:42.708911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:42.708911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:42.708911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:42.708911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:42.708911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:42.708911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:42.708911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:42.708911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:42.708911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:42.708911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:42.708911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:42.708911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:42.708911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:42.708911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:42.708911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:42.710939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:23:42.710939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:23:42.710939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:23:42.710939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:23:42.710939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:23:42.710939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:23:42.710939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:23:42.710939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:23:42.710939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 608/1244] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:23:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:23:42.716139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b6f75-3604-4120-ad7c-cb3f6f3c650d None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:42.826478 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b6f75-3604-4120-ad7c-cb3f6f3c650d None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:42.864586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c1fef-9a34-410e-a179-8040cde5bdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:42.865491 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c1fef-9a34-410e-a179-8040cde5bdb3 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:42.866076 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c1fef-9a34-410e-a179-8040cde5bdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:42.866439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c1fef-9a34-410e-a179-8040cde5bdb3 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:42.867657 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c1fef-9a34-410e-a179-8040cde5bdb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:42.868197 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c1fef-9a34-410e-a179-8040cde5bdb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:42.871327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c1fef-9a34-410e-a179-8040cde5bdb3 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:42.887389 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 637/1245] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:23:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:42.894979 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9bff46-2cfc-4a46-bf70-ed3cc5933320 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:43.012980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9bff46-2cfc-4a46-bf70-ed3cc5933320 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:43.048547 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb865a23-bf4c-4bce-af69-ab4c5146e8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:43.049499 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb865a23-bf4c-4bce-af69-ab4c5146e8cd None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:43.050000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb865a23-bf4c-4bce-af69-ab4c5146e8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:43.050444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb865a23-bf4c-4bce-af69-ab4c5146e8cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:43.051482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb865a23-bf4c-4bce-af69-ab4c5146e8cd None admin] RBAC: Adding query filter params (role.id=091d39715077464daa4cf890db55e6b9, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=ef57a9394b114eb3ab499133b5d40781) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:43.051797 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb865a23-bf4c-4bce-af69-ab4c5146e8cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:43.053449 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb865a23-bf4c-4bce-af69-ab4c5146e8cd None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:43.061934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 609/1246] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:23:42 2022] GET /identity/v3/role_assignments?user.id=ef57a9394b114eb3ab499133b5d40781&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=091d39715077464daa4cf890db55e6b9 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:23:45.325070 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a2856-128d-4a9d-b495-d60187766370 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:45.325698 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a2856-128d-4a9d-b495-d60187766370 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:45.325698 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a2856-128d-4a9d-b495-d60187766370 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:45.327125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 638/1247] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:23:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:23:45.332989 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bcaa1-1e51-4756-9dd5-fc51b03cd704 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:45.333539 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bcaa1-1e51-4756-9dd5-fc51b03cd704 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:45.333641 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bcaa1-1e51-4756-9dd5-fc51b03cd704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:45.341594 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:45.357223 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6b8bcaa1-1e51-4756-9dd5-fc51b03cd704 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:45.358444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6b8bcaa1-1e51-4756-9dd5-fc51b03cd704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:45.439967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 610/1248] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:23:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:23:45.462939 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a2856-128d-4a9d-b495-d60187766370 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:45.487113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:45.606481 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a2856-128d-4a9d-b495-d60187766370 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:45.644022 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03994152-552f-42cd-9639-179bc710d1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:45.644676 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03994152-552f-42cd-9639-179bc710d1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:45.644954 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03994152-552f-42cd-9639-179bc710d1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:45.645245 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03994152-552f-42cd-9639-179bc710d1b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:45.657502 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03994152-552f-42cd-9639-179bc710d1b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:23:45.667665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-03994152-552f-42cd-9639-179bc710d1b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:23:45.667665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:45.667665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:45.667665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:45.667665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:45.667665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:45.667665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:45.667665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:45.667665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:45.667665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:45.667665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:45.667665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:45.667665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:23:45.667665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:23:45.667665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:23:45.667665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:23:45.668951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:23:45.668951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:23:45.668951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:23:45.668951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:23:45.668951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:45.668951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:45.668951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:45.668951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:45.668951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:45.668951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:45.668951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:45.668951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:45.668951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:45.668951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:45.668951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:45.668951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:45.670106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:45.670106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:45.670106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:45.670106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:45.670106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:23:45.670106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:23:45.670106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:23:45.670106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:23:45.670106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:23:45.670106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:23:45.670106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:23:45.670106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:23:45.670106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 639/1249] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:23:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:45.679091 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bcaa1-1e51-4756-9dd5-fc51b03cd704 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:45.808669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bcaa1-1e51-4756-9dd5-fc51b03cd704 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:45.844331 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:45.852182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c2c68-b20c-4eea-a778-22cf7192a074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:45.852838 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c2c68-b20c-4eea-a778-22cf7192a074 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:45.853243 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c2c68-b20c-4eea-a778-22cf7192a074 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:45.853525 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c2c68-b20c-4eea-a778-22cf7192a074 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:45.854443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c2c68-b20c-4eea-a778-22cf7192a074 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:45.854752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c2c68-b20c-4eea-a778-22cf7192a074 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:45.856405 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c2c68-b20c-4eea-a778-22cf7192a074 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:45.867724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 611/1250] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:23:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:45.873638 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03994152-552f-42cd-9639-179bc710d1b1 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:45.970876 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:46.005580 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03994152-552f-42cd-9639-179bc710d1b1 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:46.055320 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478b49c-09c9-4945-ae29-399564b0b7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:46.055509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478b49c-09c9-4945-ae29-399564b0b7fa None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:46.055792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478b49c-09c9-4945-ae29-399564b0b7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:46.056063 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478b49c-09c9-4945-ae29-399564b0b7fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:46.070403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478b49c-09c9-4945-ae29-399564b0b7fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:46.071886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478b49c-09c9-4945-ae29-399564b0b7fa None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:46.090288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-9478b49c-09c9-4945-ae29-399564b0b7fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:46.090288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:46.090288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:46.090288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:46.090288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:46.090288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:46.090288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:46.090288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:46.090288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:46.090288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:46.090288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:46.090288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:46.090288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:23:46.090288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:23:46.090288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:23:46.092014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:23:46.092014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:46.092014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:46.092014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:46.092014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:46.092014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:46.092014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:46.092014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:46.092014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:46.092014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:46.092014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:46.092014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:46.092014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:46.092014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:46.092014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:46.092014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:46.092014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:46.093214 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:23:46.093214 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:23:46.093214 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:46.093214 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:23:46.093214 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 640/1251] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:23:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:46.097858 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c2c68-b20c-4eea-a778-22cf7192a074 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:46.220157 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c2c68-b20c-4eea-a778-22cf7192a074 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:46.261839 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8496483-373d-48b6-a16c-74a6288d2bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:46.262609 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8496483-373d-48b6-a16c-74a6288d2bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:46.262921 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8496483-373d-48b6-a16c-74a6288d2bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:46.263195 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8496483-373d-48b6-a16c-74a6288d2bb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:46.264123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8496483-373d-48b6-a16c-74a6288d2bb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:46.264532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8496483-373d-48b6-a16c-74a6288d2bb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:46.266226 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8496483-373d-48b6-a16c-74a6288d2bb2 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:46.279535 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 612/1252] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:46.285857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478b49c-09c9-4945-ae29-399564b0b7fa None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:46.411266 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478b49c-09c9-4945-ae29-399564b0b7fa None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:46.441848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:46.451924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cda5d-7f85-45fd-b193-6efabd0a95f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:46.452639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cda5d-7f85-45fd-b193-6efabd0a95f1 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:46.452951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cda5d-7f85-45fd-b193-6efabd0a95f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:46.453219 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cda5d-7f85-45fd-b193-6efabd0a95f1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:46.462255 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cda5d-7f85-45fd-b193-6efabd0a95f1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:46.463909 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cda5d-7f85-45fd-b193-6efabd0a95f1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:46.478526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-831cda5d-7f85-45fd-b193-6efabd0a95f1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:23:46.478526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:46.478526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:46.478526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:46.478526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:46.478526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:46.478526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:46.478526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:46.478526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:46.478526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:46.478526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:46.478526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:46.478526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:23:46.478526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:23:46.478526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:23:46.478526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:23:46.479886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:46.479886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:46.479886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:23:46.479886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:23:46.479886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:23:46.479886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:23:46.479886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:46.479886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:46.479886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:46.479886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:46.479886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:46.479886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:46.479886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:46.479886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:46.479886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:46.479886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:46.479886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:46.479886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:46.480933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:46.480933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:46.480933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:23:46.480933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:23:46.480933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:23:46.480933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:23:46.480933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:23:46.480933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:23:46.480933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:23:46.480933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:23:46.480933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 641/1253] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:23:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:46.485740 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8496483-373d-48b6-a16c-74a6288d2bb2 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:46.555588 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:46.607402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8496483-373d-48b6-a16c-74a6288d2bb2 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:46.646742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad0fa0-bc10-46e2-9f36-a25b466ad1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:46.647982 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad0fa0-bc10-46e2-9f36-a25b466ad1ea None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:46.648303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad0fa0-bc10-46e2-9f36-a25b466ad1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:46.648646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad0fa0-bc10-46e2-9f36-a25b466ad1ea None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:46.649794 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad0fa0-bc10-46e2-9f36-a25b466ad1ea None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:46.650049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad0fa0-bc10-46e2-9f36-a25b466ad1ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:46.651728 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad0fa0-bc10-46e2-9f36-a25b466ad1ea None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:46.677449 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 613/1254] 10.222.0.83 () {56 vars in 1143 bytes} [Mon Aug 1 15:23:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:46.678927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cda5d-7f85-45fd-b193-6efabd0a95f1 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:46.791977 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cda5d-7f85-45fd-b193-6efabd0a95f1 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:46.830106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc863fb-f2a7-49a9-b06c-bbaaf2792bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:46.830766 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc863fb-f2a7-49a9-b06c-bbaaf2792bcb None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:46.831089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc863fb-f2a7-49a9-b06c-bbaaf2792bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:46.831334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc863fb-f2a7-49a9-b06c-bbaaf2792bcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:46.906357 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:46.920691 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc863fb-f2a7-49a9-b06c-bbaaf2792bcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:46.922175 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc863fb-f2a7-49a9-b06c-bbaaf2792bcb None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:47.026939 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 642/1255] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:23:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 15:23:47.033069 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad0fa0-bc10-46e2-9f36-a25b466ad1ea None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:47.152084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad0fa0-bc10-46e2-9f36-a25b466ad1ea None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:47.170878 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:47.200231 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442a01e-2259-46fe-a59f-44c2564d217b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:47.200939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442a01e-2259-46fe-a59f-44c2564d217b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:47.201322 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442a01e-2259-46fe-a59f-44c2564d217b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:47.201678 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442a01e-2259-46fe-a59f-44c2564d217b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:47.213233 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442a01e-2259-46fe-a59f-44c2564d217b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:47.214786 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442a01e-2259-46fe-a59f-44c2564d217b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:47.232196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-a442a01e-2259-46fe-a59f-44c2564d217b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:47.232196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:47.232196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:47.232196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:47.232196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:47.232196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:47.232196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:47.232196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:47.232196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:47.232196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:47.232196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:47.232196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:47.232196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:23:47.232196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:23:47.232196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:23:47.233326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:23:47.233326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:47.233326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:47.233326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:47.233326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:47.233326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:47.233326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:47.233326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:47.233326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:47.233326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:47.233326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:47.233326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:47.233326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:47.233326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:47.233326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:47.233326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:47.233326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:47.234491 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:23:47.234491 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:23:47.234491 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:47.234491 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:23:47.234491 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 614/1256] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:23:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:47.241437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc863fb-f2a7-49a9-b06c-bbaaf2792bcb None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:47.339669 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:47.353804 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc863fb-f2a7-49a9-b06c-bbaaf2792bcb None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:47.395949 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f268e-53f9-4b85-ae4f-0c392946c24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:47.396664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f268e-53f9-4b85-ae4f-0c392946c24a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:47.396994 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f268e-53f9-4b85-ae4f-0c392946c24a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:47.397338 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f268e-53f9-4b85-ae4f-0c392946c24a None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:47.398150 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f268e-53f9-4b85-ae4f-0c392946c24a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:47.398463 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f268e-53f9-4b85-ae4f-0c392946c24a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:47.400113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f268e-53f9-4b85-ae4f-0c392946c24a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:47.412609 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 643/1257] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:47.418972 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442a01e-2259-46fe-a59f-44c2564d217b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:47.538550 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442a01e-2259-46fe-a59f-44c2564d217b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:47.585175 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ac32b-cef2-4d63-9fd7-44cc2a26d82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:47.585825 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ac32b-cef2-4d63-9fd7-44cc2a26d82e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:47.586164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ac32b-cef2-4d63-9fd7-44cc2a26d82e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:47.586438 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ac32b-cef2-4d63-9fd7-44cc2a26d82e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:47.593637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ac32b-cef2-4d63-9fd7-44cc2a26d82e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:47.595095 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ac32b-cef2-4d63-9fd7-44cc2a26d82e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:47.607365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-fd4ac32b-cef2-4d63-9fd7-44cc2a26d82e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:23:47.607365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:47.607365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:47.607365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:47.607365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:47.607365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:47.607365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:47.607365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:47.607365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:47.607365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:47.607365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:47.607365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:47.607365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:23:47.607365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:23:47.607365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:23:47.608599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:23:47.608599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:47.608599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:47.608599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:47.608599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:47.608599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:47.608599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:47.608599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:47.608599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:47.608599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:47.608599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:47.608599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:47.608599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:47.608599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:47.608599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:47.608599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:47.608599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:47.609516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:23:47.609516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:23:47.609516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:23:47.609516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:23:47.609516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:23:47.609516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:23:47.609516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:23:47.609516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:23:47.609516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 615/1258] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:23:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:47.615027 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f268e-53f9-4b85-ae4f-0c392946c24a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:47.727901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f268e-53f9-4b85-ae4f-0c392946c24a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:47.765679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4604c8b-e8a1-4263-a23e-aa05496accb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:47.766380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4604c8b-e8a1-4263-a23e-aa05496accb7 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:47.766684 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4604c8b-e8a1-4263-a23e-aa05496accb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:47.766998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4604c8b-e8a1-4263-a23e-aa05496accb7 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:47.767809 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4604c8b-e8a1-4263-a23e-aa05496accb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:47.768176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4604c8b-e8a1-4263-a23e-aa05496accb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:47.769713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4604c8b-e8a1-4263-a23e-aa05496accb7 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:47.783694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 644/1259] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:23:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:47.796881 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ac32b-cef2-4d63-9fd7-44cc2a26d82e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:47.825789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:47.915790 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ac32b-cef2-4d63-9fd7-44cc2a26d82e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:47.960696 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad8ad2-01df-4444-99c5-02bc75068c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:47.961328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad8ad2-01df-4444-99c5-02bc75068c5b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:47.961651 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad8ad2-01df-4444-99c5-02bc75068c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:47.962034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad8ad2-01df-4444-99c5-02bc75068c5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:47.962945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad8ad2-01df-4444-99c5-02bc75068c5b None admin] RBAC: Adding query filter params (role.id=7bdfa43dc1d046f7965ffe52b74bc58f, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=ef57a9394b114eb3ab499133b5d40781) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:47.963329 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad8ad2-01df-4444-99c5-02bc75068c5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:47.965236 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad8ad2-01df-4444-99c5-02bc75068c5b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:47.972458 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 616/1260] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:23:47 2022] GET /identity/v3/role_assignments?user.id=ef57a9394b114eb3ab499133b5d40781&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=7bdfa43dc1d046f7965ffe52b74bc58f => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:50.013899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4731f8-5e2f-4fd4-a05f-69ff693e579d None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:50.014451 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4731f8-5e2f-4fd4-a05f-69ff693e579d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:50.014451 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4731f8-5e2f-4fd4-a05f-69ff693e579d None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:50.015829 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 645/1261] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:23:50.020921 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6a455-44fb-4335-99f0-c88869410bbe None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:50.021381 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6a455-44fb-4335-99f0-c88869410bbe None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:50.021530 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6a455-44fb-4335-99f0-c88869410bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:50.040270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-07c6a455-44fb-4335-99f0-c88869410bbe None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:50.041464 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-07c6a455-44fb-4335-99f0-c88869410bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:50.109793 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 617/1262] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:23:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:23:50.131995 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4731f8-5e2f-4fd4-a05f-69ff693e579d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:50.142752 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:50.234462 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4731f8-5e2f-4fd4-a05f-69ff693e579d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:50.270610 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952fd76-f9cd-4a6f-9f0a-7204f1592a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:50.271181 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952fd76-f9cd-4a6f-9f0a-7204f1592a50 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:50.271463 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952fd76-f9cd-4a6f-9f0a-7204f1592a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:50.271742 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952fd76-f9cd-4a6f-9f0a-7204f1592a50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:50.283228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a952fd76-f9cd-4a6f-9f0a-7204f1592a50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:23:50.289414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-a952fd76-f9cd-4a6f-9f0a-7204f1592a50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:23:50.289414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:50.289414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:50.289414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:50.289414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:50.289414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:50.289414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:50.289414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:50.289414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:50.289414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:50.289414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:50.289414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:50.289414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:23:50.289414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:23:50.289414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:23:50.289414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:23:50.290605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:23:50.290605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:23:50.290605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:23:50.290605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:23:50.290605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:50.290605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:50.290605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:50.290605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:50.290605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:50.290605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:50.290605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:50.290605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:50.290605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:50.290605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:50.290605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:50.290605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:50.291773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:50.291773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:50.291773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:50.291773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:50.291773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:23:50.291773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:23:50.291773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:23:50.291773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:23:50.291773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:23:50.291773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:23:50.291773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:23:50.291773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:23:50.291773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 646/1263] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:23:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:50.296749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6a455-44fb-4335-99f0-c88869410bbe None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:50.315241 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:50.395872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6a455-44fb-4335-99f0-c88869410bbe None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:50.430050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179f1a1-b94f-424f-82d8-e905a81dd4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:50.430669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179f1a1-b94f-424f-82d8-e905a81dd4fb None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:50.430952 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179f1a1-b94f-424f-82d8-e905a81dd4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:50.431228 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179f1a1-b94f-424f-82d8-e905a81dd4fb None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:50.431942 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179f1a1-b94f-424f-82d8-e905a81dd4fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:50.432250 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179f1a1-b94f-424f-82d8-e905a81dd4fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:50.433621 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179f1a1-b94f-424f-82d8-e905a81dd4fb None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:50.441508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 618/1264] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:23:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:50.449790 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952fd76-f9cd-4a6f-9f0a-7204f1592a50 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:50.519467 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:50.560558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952fd76-f9cd-4a6f-9f0a-7204f1592a50 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:50.600736 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bff498-8f32-436f-b27b-7f245b87bcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:50.601378 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bff498-8f32-436f-b27b-7f245b87bcee None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:50.601654 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bff498-8f32-436f-b27b-7f245b87bcee None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:50.602040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bff498-8f32-436f-b27b-7f245b87bcee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:50.612552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bff498-8f32-436f-b27b-7f245b87bcee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:50.614070 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bff498-8f32-436f-b27b-7f245b87bcee None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:50.629512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-49bff498-8f32-436f-b27b-7f245b87bcee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:50.629512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:50.629512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:50.629512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:50.629512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:50.629512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:50.629512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:50.629512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:50.629512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:50.629512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:50.629512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:50.629512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:50.629512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:23:50.629512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:23:50.629512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:23:50.630666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:23:50.630666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:50.630666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:50.630666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:50.630666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:50.630666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:50.630666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:50.630666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:50.630666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:50.630666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:50.630666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:50.630666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:50.630666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:50.630666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:50.630666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:50.630666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:50.630666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:50.632313 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:23:50.632313 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:23:50.632313 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:50.632313 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:23:50.632313 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 647/1265] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:23:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:50.637110 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179f1a1-b94f-424f-82d8-e905a81dd4fb None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:50.718554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:50.737325 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179f1a1-b94f-424f-82d8-e905a81dd4fb None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:50.771606 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724041cd-a587-4ed1-9656-d3b3befb62d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:50.772756 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724041cd-a587-4ed1-9656-d3b3befb62d6 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:50.773040 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724041cd-a587-4ed1-9656-d3b3befb62d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:50.773314 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724041cd-a587-4ed1-9656-d3b3befb62d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:50.774031 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724041cd-a587-4ed1-9656-d3b3befb62d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:50.774335 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724041cd-a587-4ed1-9656-d3b3befb62d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:50.775722 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724041cd-a587-4ed1-9656-d3b3befb62d6 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:50.787453 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 619/1266] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:50.793132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bff498-8f32-436f-b27b-7f245b87bcee None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:50.891899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bff498-8f32-436f-b27b-7f245b87bcee None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:50.912805 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:50.926895 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5861a-c58c-4ec8-92d2-25bc34f8f6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:50.927537 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5861a-c58c-4ec8-92d2-25bc34f8f6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:50.927817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5861a-c58c-4ec8-92d2-25bc34f8f6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:50.928643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5861a-c58c-4ec8-92d2-25bc34f8f6b9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:50.936018 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5861a-c58c-4ec8-92d2-25bc34f8f6b9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:50.937456 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5861a-c58c-4ec8-92d2-25bc34f8f6b9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:50.949574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-3de5861a-c58c-4ec8-92d2-25bc34f8f6b9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:23:50.949574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:50.949574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:50.949574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:50.949574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:50.949574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:50.949574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:50.949574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:50.949574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:50.949574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:50.949574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:50.949574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:50.949574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:23:50.949574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:23:50.949574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:23:50.949574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:23:50.950775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:50.950775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:50.950775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:23:50.950775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:23:50.950775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:23:50.950775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:23:50.950775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:50.950775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:50.950775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:50.950775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:50.950775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:50.950775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:50.950775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:50.950775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:50.950775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:50.950775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:50.950775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:50.950775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:50.951968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:50.951968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:50.951968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:23:50.951968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:23:50.951968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:23:50.951968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:23:50.951968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:23:50.951968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:23:50.951968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:23:50.951968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:23:50.951968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 648/1267] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:23:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:50.956644 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724041cd-a587-4ed1-9656-d3b3befb62d6 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:51.058338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724041cd-a587-4ed1-9656-d3b3befb62d6 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:51.093016 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87d269-6eff-4594-9ebc-f37ae65b04a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:51.093627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87d269-6eff-4594-9ebc-f37ae65b04a5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:51.093924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87d269-6eff-4594-9ebc-f37ae65b04a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:51.094202 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87d269-6eff-4594-9ebc-f37ae65b04a5 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:51.094987 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87d269-6eff-4594-9ebc-f37ae65b04a5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:51.095296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87d269-6eff-4594-9ebc-f37ae65b04a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:51.096724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87d269-6eff-4594-9ebc-f37ae65b04a5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:51.105136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:51.113675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 620/1268] 10.222.0.83 () {56 vars in 1143 bytes} [Mon Aug 1 15:23:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:51.120295 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5861a-c58c-4ec8-92d2-25bc34f8f6b9 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:51.220357 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5861a-c58c-4ec8-92d2-25bc34f8f6b9 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:51.254326 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32484fec-4b8f-4848-9261-688e7452853d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:51.254970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32484fec-4b8f-4848-9261-688e7452853d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:51.255253 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32484fec-4b8f-4848-9261-688e7452853d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:51.255528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32484fec-4b8f-4848-9261-688e7452853d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:51.266158 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32484fec-4b8f-4848-9261-688e7452853d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:51.267601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32484fec-4b8f-4848-9261-688e7452853d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:51.281618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-32484fec-4b8f-4848-9261-688e7452853d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:51.281618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:51.281618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:51.281618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:51.281618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:51.281618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:51.281618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:51.281618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:51.281618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:51.281618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:51.281618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:51.281618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:51.281618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:23:51.281618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:23:51.281618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:23:51.282738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:23:51.282738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:51.282738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:51.282738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:51.282738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:51.282738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:51.282738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:51.282738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:51.282738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:51.282738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:51.282738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:51.282738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:51.282738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:51.282738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:51.282738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:51.282738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:51.282738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:51.283792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:23:51.283792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:23:51.283792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:51.283792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:23:51.283792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 649/1269] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:23:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:51.289026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87d269-6eff-4594-9ebc-f37ae65b04a5 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:51.387491 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87d269-6eff-4594-9ebc-f37ae65b04a5 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:51.420827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2d3cb-0529-4898-abb9-8526348fb28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:51.421440 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2d3cb-0529-4898-abb9-8526348fb28d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:51.421727 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2d3cb-0529-4898-abb9-8526348fb28d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:51.422033 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2d3cb-0529-4898-abb9-8526348fb28d None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:51.422724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2d3cb-0529-4898-abb9-8526348fb28d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:51.423033 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2d3cb-0529-4898-abb9-8526348fb28d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:51.424412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2d3cb-0529-4898-abb9-8526348fb28d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:51.435950 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 621/1270] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:51.441756 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32484fec-4b8f-4848-9261-688e7452853d None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:51.473453 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:51.552823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32484fec-4b8f-4848-9261-688e7452853d None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:51.587116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49754d51-b0ec-4119-8c36-0d905fd4738a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:51.587744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49754d51-b0ec-4119-8c36-0d905fd4738a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:51.588026 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49754d51-b0ec-4119-8c36-0d905fd4738a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:51.588300 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49754d51-b0ec-4119-8c36-0d905fd4738a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:51.594826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49754d51-b0ec-4119-8c36-0d905fd4738a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:51.596262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49754d51-b0ec-4119-8c36-0d905fd4738a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:51.607477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-49754d51-b0ec-4119-8c36-0d905fd4738a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:23:51.607477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:51.607477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:51.607477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:51.607477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:51.607477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:51.607477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:51.607477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:51.607477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:51.607477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:51.607477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:51.607477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:51.607477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:23:51.607477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:23:51.607477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:23:51.608390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:23:51.608390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:51.608390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:51.608390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:51.608390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:51.608390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:51.608390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:51.608390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:51.608390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:51.608390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:51.608390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:51.608390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:51.608390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:51.608390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:51.608390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:51.608390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:51.608390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:51.609368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:23:51.609368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:23:51.609368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:23:51.609368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:23:51.609368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:23:51.609368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:23:51.609368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:23:51.609368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:23:51.609368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 650/1271] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:23:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:51.614786 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2d3cb-0529-4898-abb9-8526348fb28d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:51.673749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:51.715914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2d3cb-0529-4898-abb9-8526348fb28d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:51.749082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98474b8-393d-418a-af9c-a4c3f0e00ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:51.749631 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98474b8-393d-418a-af9c-a4c3f0e00ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:51.749943 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98474b8-393d-418a-af9c-a4c3f0e00ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:51.750227 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98474b8-393d-418a-af9c-a4c3f0e00ac1 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:51.750985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98474b8-393d-418a-af9c-a4c3f0e00ac1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:51.751319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98474b8-393d-418a-af9c-a4c3f0e00ac1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:51.752761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98474b8-393d-418a-af9c-a4c3f0e00ac1 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:51.765148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 622/1272] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:23:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:51.771166 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49754d51-b0ec-4119-8c36-0d905fd4738a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:51.860854 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:51.870578 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49754d51-b0ec-4119-8c36-0d905fd4738a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:51.904165 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacdc01-5446-40bd-a9c5-c76872732ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:51.904835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacdc01-5446-40bd-a9c5-c76872732ab9 None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:51.905119 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacdc01-5446-40bd-a9c5-c76872732ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:51.905392 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacdc01-5446-40bd-a9c5-c76872732ab9 None admin] PATH_INFO: `/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/ef57a9394b114eb3ab499133b5d40781/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:51.907374 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfacdc01-5446-40bd-a9c5-c76872732ab9 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f4e27cc50bd4f9f8ced4cc544ab3064, user_id=ef57a9394b114eb3ab499133b5d40781, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:51.908812 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfacdc01-5446-40bd-a9c5-c76872732ab9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:51.922982 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 651/1273] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Aug 1 15:23:51 2022] PUT /identity/v3/projects/1f4e27cc50bd4f9f8ced4cc544ab3064/users/ef57a9394b114eb3ab499133b5d40781/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:53.915163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dd58f-21e7-4f83-839a-010d523febae None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:53.915698 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dd58f-21e7-4f83-839a-010d523febae None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:53.915698 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dd58f-21e7-4f83-839a-010d523febae None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:53.917092 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 623/1274] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:23:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:23:53.922529 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225c1cd-6b98-466e-9040-1e3d805f15c2 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:53.922946 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225c1cd-6b98-466e-9040-1e3d805f15c2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:53.923087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225c1cd-6b98-466e-9040-1e3d805f15c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:53.940855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e225c1cd-6b98-466e-9040-1e3d805f15c2 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:53.942215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e225c1cd-6b98-466e-9040-1e3d805f15c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:54.007245 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 652/1275] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:23:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:23:54.030576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dd58f-21e7-4f83-839a-010d523febae None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:54.136215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dd58f-21e7-4f83-839a-010d523febae None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:54.154251 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:54.172356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f44ec-104e-408e-9166-d05a62de4e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:54.172935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f44ec-104e-408e-9166-d05a62de4e40 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:54.173219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f44ec-104e-408e-9166-d05a62de4e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:54.173496 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f44ec-104e-408e-9166-d05a62de4e40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:54.183677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f44ec-104e-408e-9166-d05a62de4e40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:23:54.188645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-1d9f44ec-104e-408e-9166-d05a62de4e40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:23:54.188645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:54.188645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:54.188645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:54.188645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:54.188645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:54.188645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:54.188645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:54.188645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:54.188645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:54.188645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:54.188645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:54.188645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:23:54.188645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:23:54.188645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:23:54.188645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:23:54.189799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:23:54.189799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:23:54.189799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:23:54.189799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:23:54.189799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:54.189799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:54.189799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:54.189799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:54.189799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:54.189799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:54.189799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:54.189799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:54.189799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:54.189799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:54.189799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:54.189799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:54.190821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:54.190821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:54.190821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:54.190821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:54.190821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:23:54.190821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:23:54.190821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:23:54.190821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:23:54.190821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:23:54.190821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:23:54.190821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:23:54.190821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:23:54.190821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 624/1276] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Aug 1 15:23:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:54.196630 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225c1cd-6b98-466e-9040-1e3d805f15c2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:54.276679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:54.294884 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225c1cd-6b98-466e-9040-1e3d805f15c2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:54.328461 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d052103-587a-49ab-b260-5518be59303f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:54.329081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d052103-587a-49ab-b260-5518be59303f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:54.329359 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d052103-587a-49ab-b260-5518be59303f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:54.329632 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d052103-587a-49ab-b260-5518be59303f None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:54.330383 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d052103-587a-49ab-b260-5518be59303f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:54.330689 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d052103-587a-49ab-b260-5518be59303f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:54.332060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d052103-587a-49ab-b260-5518be59303f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:54.340497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 653/1277] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Aug 1 15:23:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:54.346385 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f44ec-104e-408e-9166-d05a62de4e40 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:54.451230 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f44ec-104e-408e-9166-d05a62de4e40 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:54.484302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07834de-0083-4c1b-bc6c-34c5fc1bba7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:54.484949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07834de-0083-4c1b-bc6c-34c5fc1bba7b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:54.485235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07834de-0083-4c1b-bc6c-34c5fc1bba7b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:54.485511 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07834de-0083-4c1b-bc6c-34c5fc1bba7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:54.496515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07834de-0083-4c1b-bc6c-34c5fc1bba7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:54.498007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07834de-0083-4c1b-bc6c-34c5fc1bba7b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:54.513415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-d07834de-0083-4c1b-bc6c-34c5fc1bba7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:54.513415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:54.513415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:54.513415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:54.513415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:54.513415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:54.513415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:54.513415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:54.513415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:54.513415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:54.513415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:54.513415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:54.513415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:23:54.513415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:23:54.513415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:23:54.514419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:23:54.514419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:54.514419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:54.514419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:54.514419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:54.514419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:54.514419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:54.514419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:54.514419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:54.514419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:54.514419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:54.514419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:54.514419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:54.514419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:54.514419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:54.514419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:54.514419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:54.515390 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:23:54.515390 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:23:54.515390 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:54.515390 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:23:54.515390 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 625/1278] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:23:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:54.520666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d052103-587a-49ab-b260-5518be59303f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:54.631365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d052103-587a-49ab-b260-5518be59303f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:54.664870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fe56b-135b-4a87-a3d1-f2766b3f7f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:54.665482 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fe56b-135b-4a87-a3d1-f2766b3f7f3e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:54.665760 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fe56b-135b-4a87-a3d1-f2766b3f7f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:54.666063 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fe56b-135b-4a87-a3d1-f2766b3f7f3e None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:54.666744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7fe56b-135b-4a87-a3d1-f2766b3f7f3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:54.667045 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7fe56b-135b-4a87-a3d1-f2766b3f7f3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:54.668411 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7fe56b-135b-4a87-a3d1-f2766b3f7f3e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:54.680370 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 654/1279] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:54.686323 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07834de-0083-4c1b-bc6c-34c5fc1bba7b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:54.719229 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:54.793543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07834de-0083-4c1b-bc6c-34c5fc1bba7b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:54.827426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee7ea2-8129-4d7a-b264-1c38bf235958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:54.828063 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee7ea2-8129-4d7a-b264-1c38bf235958 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:54.828346 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee7ea2-8129-4d7a-b264-1c38bf235958 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:54.828620 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee7ea2-8129-4d7a-b264-1c38bf235958 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:54.835943 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee7ea2-8129-4d7a-b264-1c38bf235958 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:54.837391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee7ea2-8129-4d7a-b264-1c38bf235958 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:54.848379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-47ee7ea2-8129-4d7a-b264-1c38bf235958 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:23:54.848379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:54.848379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:54.848379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:54.848379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:54.848379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:54.848379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:54.848379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:54.848379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:54.848379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:54.848379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:54.848379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:54.848379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:23:54.848379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:23:54.848379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:23:54.848379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:23:54.849509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:54.849509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:54.849509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:23:54.849509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:23:54.849509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:23:54.849509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:23:54.849509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:54.849509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:54.849509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:54.849509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:54.849509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:54.849509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:54.849509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:54.849509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:54.849509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:54.849509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:54.849509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:54.849509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:54.850860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:54.850860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:54.850860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:23:54.850860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:23:54.850860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:23:54.850860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:23:54.850860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:23:54.850860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:23:54.850860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:23:54.850860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:23:54.850860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 626/1280] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:23:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:54.855489 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fe56b-135b-4a87-a3d1-f2766b3f7f3e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:54.859261 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:54.962301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fe56b-135b-4a87-a3d1-f2766b3f7f3e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:54.996166 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea81562-6680-44c7-8f9b-2ade858e8bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:54.996722 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea81562-6680-44c7-8f9b-2ade858e8bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:54.997005 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea81562-6680-44c7-8f9b-2ade858e8bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:54.997281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea81562-6680-44c7-8f9b-2ade858e8bf4 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:54.998120 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea81562-6680-44c7-8f9b-2ade858e8bf4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:54.998429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea81562-6680-44c7-8f9b-2ade858e8bf4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:54.999861 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea81562-6680-44c7-8f9b-2ade858e8bf4 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:55.017343 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 655/1281] 10.222.0.83 () {56 vars in 1143 bytes} [Mon Aug 1 15:23:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:55.023322 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee7ea2-8129-4d7a-b264-1c38bf235958 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:55.125172 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:55.139277 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee7ea2-8129-4d7a-b264-1c38bf235958 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:55.180478 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a29e1c-3230-459c-b306-447a2421ed56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:55.181092 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a29e1c-3230-459c-b306-447a2421ed56 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:55.181470 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a29e1c-3230-459c-b306-447a2421ed56 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:55.181881 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a29e1c-3230-459c-b306-447a2421ed56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:55.250058 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a29e1c-3230-459c-b306-447a2421ed56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:55.251451 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a29e1c-3230-459c-b306-447a2421ed56 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:55.342177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:55.352863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 627/1282] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Aug 1 15:23:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:23:55.358832 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea81562-6680-44c7-8f9b-2ade858e8bf4 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:55.418958 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:55.457763 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea81562-6680-44c7-8f9b-2ade858e8bf4 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:55.492604 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f485bb-646c-4632-a802-b6d05988be1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:55.493243 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f485bb-646c-4632-a802-b6d05988be1c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:55.493523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f485bb-646c-4632-a802-b6d05988be1c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:55.493795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f485bb-646c-4632-a802-b6d05988be1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:55.503891 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f485bb-646c-4632-a802-b6d05988be1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:55.505272 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f485bb-646c-4632-a802-b6d05988be1c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:55.522473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-69f485bb-646c-4632-a802-b6d05988be1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:55.522473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:55.522473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:55.522473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:55.522473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:55.522473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:55.522473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:55.522473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:55.522473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:55.522473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:55.522473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:55.522473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:55.522473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:23:55.522473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:23:55.522473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:23:55.523636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:23:55.523636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:55.523636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:55.523636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:55.523636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:55.523636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:55.523636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:55.523636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:55.523636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:55.523636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:55.523636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:55.523636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:55.523636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:55.523636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:55.523636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:55.523636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:55.523636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:55.524719 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:23:55.524719 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:23:55.524719 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:23:55.524719 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:23:55.524719 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 656/1283] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:23:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:55.529667 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a29e1c-3230-459c-b306-447a2421ed56 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:55.632925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a29e1c-3230-459c-b306-447a2421ed56 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:55.668694 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e842c1ef-99e3-4757-8423-cafbcacaae5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:55.669375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842c1ef-99e3-4757-8423-cafbcacaae5a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:55.669622 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842c1ef-99e3-4757-8423-cafbcacaae5a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:55.670074 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842c1ef-99e3-4757-8423-cafbcacaae5a None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:55.670599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e842c1ef-99e3-4757-8423-cafbcacaae5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:55.670882 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e842c1ef-99e3-4757-8423-cafbcacaae5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:55.672266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e842c1ef-99e3-4757-8423-cafbcacaae5a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:55.684220 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 628/1284] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Aug 1 15:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:23:55.689970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f485bb-646c-4632-a802-b6d05988be1c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:55.786383 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f485bb-646c-4632-a802-b6d05988be1c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:55.800470 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:55.821741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5ed65-d045-4c38-a228-3afb9d558b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:55.822415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5ed65-d045-4c38-a228-3afb9d558b93 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:55.822696 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5ed65-d045-4c38-a228-3afb9d558b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:55.822971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5ed65-d045-4c38-a228-3afb9d558b93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:55.829949 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5ed65-d045-4c38-a228-3afb9d558b93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:55.831396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5ed65-d045-4c38-a228-3afb9d558b93 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:55.842855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-94e5ed65-d045-4c38-a228-3afb9d558b93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:23:55.842855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:55.842855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:55.842855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:55.842855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:55.842855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:55.842855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:55.842855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:55.842855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:55.842855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:55.842855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:55.842855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:55.842855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:23:55.842855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:23:55.842855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:23:55.843844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:23:55.843844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:55.843844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:55.843844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:55.843844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:55.843844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:55.843844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:55.843844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:55.843844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:55.843844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:55.843844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:55.843844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:55.843844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:55.843844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:55.843844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:55.843844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:55.843844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:55.844771 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:23:55.844771 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:23:55.844771 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:23:55.844771 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:23:55.844771 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:23:55.844771 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:23:55.844771 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:23:55.844771 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 15:23:55.844771 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 657/1285] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Aug 1 15:23:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:55.850395 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e842c1ef-99e3-4757-8423-cafbcacaae5a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:55.900099 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:55.950146 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e842c1ef-99e3-4757-8423-cafbcacaae5a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:55.982960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4bd9c5-19d3-4afa-9946-390ddc1ef8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:55.983577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bd9c5-19d3-4afa-9946-390ddc1ef8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:55.983857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bd9c5-19d3-4afa-9946-390ddc1ef8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:55.984130 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bd9c5-19d3-4afa-9946-390ddc1ef8b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:55.984863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4bd9c5-19d3-4afa-9946-390ddc1ef8b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:55.985165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4bd9c5-19d3-4afa-9946-390ddc1ef8b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:55.986626 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4bd9c5-19d3-4afa-9946-390ddc1ef8b4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:55.999080 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 629/1286] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Aug 1 15:23:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:56.005253 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5ed65-d045-4c38-a228-3afb9d558b93 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:56.105971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5ed65-d045-4c38-a228-3afb9d558b93 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:56.138303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e135555-5bec-46db-9210-4cb82e9a30cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:56.138914 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e135555-5bec-46db-9210-4cb82e9a30cb None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:56.139196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e135555-5bec-46db-9210-4cb82e9a30cb None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:56.139473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e135555-5bec-46db-9210-4cb82e9a30cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:56.140255 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e135555-5bec-46db-9210-4cb82e9a30cb None admin] RBAC: Adding query filter params (role.id=7bdfa43dc1d046f7965ffe52b74bc58f, scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064, user.id=ef57a9394b114eb3ab499133b5d40781) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:56.140565 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e135555-5bec-46db-9210-4cb82e9a30cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:56.142015 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e135555-5bec-46db-9210-4cb82e9a30cb None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:56.148251 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 658/1287] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Aug 1 15:23:56 2022] GET /identity/v3/role_assignments?user.id=ef57a9394b114eb3ab499133b5d40781&scope.project.id=1f4e27cc50bd4f9f8ced4cc544ab3064&role.id=7bdfa43dc1d046f7965ffe52b74bc58f => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:58.193971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64d56b-2795-4750-9809-00faa5e42ed8 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:58.194589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64d56b-2795-4750-9809-00faa5e42ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:58.194589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64d56b-2795-4750-9809-00faa5e42ed8 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:58.195902 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 630/1288] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:23:58.201404 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30b3be-fe05-4ace-a6c7-c364edb46bd9 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:58.201833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30b3be-fe05-4ace-a6c7-c364edb46bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:58.201971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30b3be-fe05-4ace-a6c7-c364edb46bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:58.219705 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2d30b3be-fe05-4ace-a6c7-c364edb46bd9 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:58.220894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2d30b3be-fe05-4ace-a6c7-c364edb46bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:58.251110 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:58.288662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 659/1289] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:23:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:23:58.310957 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64d56b-2795-4750-9809-00faa5e42ed8 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:58.410940 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64d56b-2795-4750-9809-00faa5e42ed8 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:58.422118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:58.445811 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18e817-2a5a-474d-be18-c6c8e01410d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:58.446487 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18e817-2a5a-474d-be18-c6c8e01410d3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:58.446773 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18e817-2a5a-474d-be18-c6c8e01410d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:58.447050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18e817-2a5a-474d-be18-c6c8e01410d3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:58.447675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18e817-2a5a-474d-be18-c6c8e01410d3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:58.449306 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18e817-2a5a-474d-be18-c6c8e01410d3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:58.458109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-bd18e817-2a5a-474d-be18-c6c8e01410d3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:23:58.458109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:58.458109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:23:58.458109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:23:58.458109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:23:58.458109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:23:58.458109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:23:58.458109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:23:58.458109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:23:58.458109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:23:58.458109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:23:58.458109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:23:58.458109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:23:58.458109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:23:58.458109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:23:58.458109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:23:58.458109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:23:58.459445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:23:58.459445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:23:58.459445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:23:58.459445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:23:58.459445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:23:58.459445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:23:58.459445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:23:58.459445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:23:58.459445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:23:58.459445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:23:58.459445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:23:58.459445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:23:58.459445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:23:58.459445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:23:58.459445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:23:58.459445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:23:58.461084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:23:58.461084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:23:58.461084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:23:58.461084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:23:58.461084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:23:58.461084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:23:58.461084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:23:58.461084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:23:58.461084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:23:58.461084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:23:58.461084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:23:58.461084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:23:58.461084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:23:58.461084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:23:58.461084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 631/1290] 10.222.0.83 () {56 vars in 1117 bytes} [Mon Aug 1 15:23:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:23:58.465832 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30b3be-fe05-4ace-a6c7-c364edb46bd9 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:58.568611 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30b3be-fe05-4ace-a6c7-c364edb46bd9 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:58.608127 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a692a0-a4b1-4cc5-8099-af70d24ff304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:58.608751 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a692a0-a4b1-4cc5-8099-af70d24ff304 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:58.609112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a692a0-a4b1-4cc5-8099-af70d24ff304 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:58.609308 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a692a0-a4b1-4cc5-8099-af70d24ff304 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:58.610113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a692a0-a4b1-4cc5-8099-af70d24ff304 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:58.610475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a692a0-a4b1-4cc5-8099-af70d24ff304 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:58.611688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a692a0-a4b1-4cc5-8099-af70d24ff304 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:58.616792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 660/1291] 10.222.0.83 () {56 vars in 1116 bytes} [Mon Aug 1 15:23:58 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:58.622638 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18e817-2a5a-474d-be18-c6c8e01410d3 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:58.720441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18e817-2a5a-474d-be18-c6c8e01410d3 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:58.752327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d872d6-af65-422d-8bd2-3e11f66d7147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:58.752944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d872d6-af65-422d-8bd2-3e11f66d7147 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:58.753223 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d872d6-af65-422d-8bd2-3e11f66d7147 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:58.753495 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d872d6-af65-422d-8bd2-3e11f66d7147 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:58.754193 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d872d6-af65-422d-8bd2-3e11f66d7147 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:58.754496 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d872d6-af65-422d-8bd2-3e11f66d7147 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:58.755854 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d872d6-af65-422d-8bd2-3e11f66d7147 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:58.760738 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 632/1292] 10.222.0.83 () {56 vars in 1116 bytes} [Mon Aug 1 15:23:58 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:24:00.743933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54734f4-f728-4153-ac15-d6cc60f787b9 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:00.744562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54734f4-f728-4153-ac15-d6cc60f787b9 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:00.744562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54734f4-f728-4153-ac15-d6cc60f787b9 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:00.745845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 661/1293] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:24:00.750944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e51fd-eab7-4494-afe4-dc4d51c2588f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:00.751346 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e51fd-eab7-4494-afe4-dc4d51c2588f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:00.751501 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e51fd-eab7-4494-afe4-dc4d51c2588f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:00.769341 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-782e51fd-eab7-4494-afe4-dc4d51c2588f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:00.770573 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-782e51fd-eab7-4494-afe4-dc4d51c2588f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:00.803655 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:00.841543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 633/1294] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:24:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:24:00.864303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54734f4-f728-4153-ac15-d6cc60f787b9 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:00.895845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:00.968202 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54734f4-f728-4153-ac15-d6cc60f787b9 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:01.000727 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd217db-89e8-46f4-af52-a36409c3dff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:01.001269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd217db-89e8-46f4-af52-a36409c3dff0 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:01.001553 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd217db-89e8-46f4-af52-a36409c3dff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:01.001828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd217db-89e8-46f4-af52-a36409c3dff0 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:01.002553 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd217db-89e8-46f4-af52-a36409c3dff0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:01.003918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd217db-89e8-46f4-af52-a36409c3dff0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:01.018648 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 662/1295] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Aug 1 15:24:00 2022] POST /identity/v3/services => generated 246 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:03.041377 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f8fe1-4feb-4517-ad4f-99253e4b8020 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:03.041956 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f8fe1-4feb-4517-ad4f-99253e4b8020 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:03.041956 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f8fe1-4feb-4517-ad4f-99253e4b8020 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:03.043375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 634/1296] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:24:03.048422 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234bbee-95e3-4179-a1c7-26cad1ced9a2 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:03.048847 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234bbee-95e3-4179-a1c7-26cad1ced9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:03.048981 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234bbee-95e3-4179-a1c7-26cad1ced9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:03.067347 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-7234bbee-95e3-4179-a1c7-26cad1ced9a2 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:03.068542 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7234bbee-95e3-4179-a1c7-26cad1ced9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:03.141310 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 663/1297] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:24:03 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:24:03.164007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f8fe1-4feb-4517-ad4f-99253e4b8020 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:03.265590 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f8fe1-4feb-4517-ad4f-99253e4b8020 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:03.299053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07662c61-25bc-4cd4-8d0d-49f59ad39768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:03.299621 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07662c61-25bc-4cd4-8d0d-49f59ad39768 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:03.299904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07662c61-25bc-4cd4-8d0d-49f59ad39768 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:03.300175 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07662c61-25bc-4cd4-8d0d-49f59ad39768 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:03.300799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07662c61-25bc-4cd4-8d0d-49f59ad39768 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:03.302199 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07662c61-25bc-4cd4-8d0d-49f59ad39768 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:03.311628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-07662c61-25bc-4cd4-8d0d-49f59ad39768 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:24:03.311628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:24:03.311628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:24:03.311628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:24:03.311628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:24:03.311628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:24:03.311628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:24:03.311628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:24:03.311628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:24:03.311628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:24:03.311628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:24:03.311628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:24:03.311628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:24:03.311628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:24:03.311628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:24:03.311628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:24:03.311628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:24:03.312817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:24:03.312817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:24:03.312817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:24:03.312817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:24:03.312817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:24:03.312817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:24:03.312817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:24:03.312817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:24:03.312817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:24:03.312817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:24:03.312817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:24:03.312817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:24:03.312817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:24:03.312817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:24:03.312817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:24:03.312817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:24:03.313692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:24:03.313692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:24:03.313692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:24:03.313692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:24:03.313692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:24:03.313692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:24:03.313692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:24:03.313692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:24:03.313692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:24:03.313692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:24:03.313692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:24:03.313692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:24:03.313692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:24:03.313692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:24:03.313692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 635/1298] 10.222.0.83 () {56 vars in 1117 bytes} [Mon Aug 1 15:24:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:24:03.319569 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234bbee-95e3-4179-a1c7-26cad1ced9a2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:03.338292 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:03.418396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234bbee-95e3-4179-a1c7-26cad1ced9a2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:03.454507 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faadc7b5-94c1-4746-86d5-e3d14739a724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:03.455535 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadc7b5-94c1-4746-86d5-e3d14739a724 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:03.455535 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadc7b5-94c1-4746-86d5-e3d14739a724 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:03.458406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadc7b5-94c1-4746-86d5-e3d14739a724 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:03.458406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faadc7b5-94c1-4746-86d5-e3d14739a724 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:24:03.458406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faadc7b5-94c1-4746-86d5-e3d14739a724 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:03.458406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faadc7b5-94c1-4746-86d5-e3d14739a724 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:03.463517 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 664/1299] 10.222.0.83 () {56 vars in 1116 bytes} [Mon Aug 1 15:24:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:24:03.469484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07662c61-25bc-4cd4-8d0d-49f59ad39768 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:03.502116 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:03.586750 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07662c61-25bc-4cd4-8d0d-49f59ad39768 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:03.625214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641eeda-9772-45c0-8387-1921d12a973f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:03.625863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641eeda-9772-45c0-8387-1921d12a973f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:03.626181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641eeda-9772-45c0-8387-1921d12a973f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:03.626458 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641eeda-9772-45c0-8387-1921d12a973f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:03.627213 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641eeda-9772-45c0-8387-1921d12a973f None admin] RBAC: Adding query filter params (interface=public, service_id=5ae226f79f53481e9fa1e588f4db6680, region_id=RegionOne) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:24:03.627523 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641eeda-9772-45c0-8387-1921d12a973f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:03.628895 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641eeda-9772-45c0-8387-1921d12a973f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:03.633926 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 636/1300] 10.222.0.83 () {56 vars in 1251 bytes} [Mon Aug 1 15:24:03 2022] GET /identity/v3/endpoints?service_id=5ae226f79f53481e9fa1e588f4db6680&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:24:03.639629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faadc7b5-94c1-4746-86d5-e3d14739a724 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:03.720473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:03.740692 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faadc7b5-94c1-4746-86d5-e3d14739a724 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:03.774122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c8f2d-094f-4919-86ac-905ddba26019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:03.774737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c8f2d-094f-4919-86ac-905ddba26019 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:03.775020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c8f2d-094f-4919-86ac-905ddba26019 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:03.775289 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c8f2d-094f-4919-86ac-905ddba26019 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:03.775933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c8f2d-094f-4919-86ac-905ddba26019 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:03.777293 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c8f2d-094f-4919-86ac-905ddba26019 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:03.783978 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 665/1301] 10.222.0.83 () {56 vars in 1087 bytes} [Mon Aug 1 15:24:03 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:05.790267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec46b0b-9205-460e-a086-81cd401b9178 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:05.790827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec46b0b-9205-460e-a086-81cd401b9178 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:05.790926 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec46b0b-9205-460e-a086-81cd401b9178 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:05.792211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 637/1302] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:24:05.798557 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877fb33-5000-40c8-bcd8-65849f22fa42 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:05.798944 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877fb33-5000-40c8-bcd8-65849f22fa42 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:05.799114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877fb33-5000-40c8-bcd8-65849f22fa42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:05.817445 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8877fb33-5000-40c8-bcd8-65849f22fa42 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:05.818778 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8877fb33-5000-40c8-bcd8-65849f22fa42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:05.888592 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 666/1303] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:24:05 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:24:05.911011 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec46b0b-9205-460e-a086-81cd401b9178 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:06.006268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:06.014733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec46b0b-9205-460e-a086-81cd401b9178 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:06.047713 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55670714-6469-45dc-8136-3b0abf3cdd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:06.048293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55670714-6469-45dc-8136-3b0abf3cdd8c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:06.048576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55670714-6469-45dc-8136-3b0abf3cdd8c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:06.048855 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55670714-6469-45dc-8136-3b0abf3cdd8c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:06.049488 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55670714-6469-45dc-8136-3b0abf3cdd8c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:06.050903 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55670714-6469-45dc-8136-3b0abf3cdd8c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:06.061007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-55670714-6469-45dc-8136-3b0abf3cdd8c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:24:06.061007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:24:06.061007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:24:06.061007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:24:06.061007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:24:06.061007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:24:06.061007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:24:06.061007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:24:06.061007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:24:06.061007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:24:06.061007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:24:06.061007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:24:06.061007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:24:06.061007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:24:06.061007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:24:06.061007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:24:06.061007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:24:06.062148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:24:06.062148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:24:06.062148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:24:06.062148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:24:06.062148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:24:06.062148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:24:06.062148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:24:06.062148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:24:06.062148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:24:06.062148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:24:06.062148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:24:06.062148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:24:06.062148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:24:06.062148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:24:06.062148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:24:06.062148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:24:06.063022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:24:06.063022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:24:06.063022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:24:06.063022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:24:06.063022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:24:06.063022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:24:06.063022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:24:06.063022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:24:06.063022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:24:06.063022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:24:06.063022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:24:06.063022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:24:06.063022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:24:06.063022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:24:06.063022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 638/1304] 10.222.0.83 () {56 vars in 1117 bytes} [Mon Aug 1 15:24:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:24:06.068155 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877fb33-5000-40c8-bcd8-65849f22fa42 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:06.156741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:06.179455 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877fb33-5000-40c8-bcd8-65849f22fa42 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:06.212858 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5775c5b-f853-4911-925d-e38b1fdc179c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:06.213471 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5775c5b-f853-4911-925d-e38b1fdc179c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:06.213748 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5775c5b-f853-4911-925d-e38b1fdc179c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:06.214049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5775c5b-f853-4911-925d-e38b1fdc179c None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:06.214715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5775c5b-f853-4911-925d-e38b1fdc179c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:24:06.215014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5775c5b-f853-4911-925d-e38b1fdc179c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:06.216369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5775c5b-f853-4911-925d-e38b1fdc179c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:06.221332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 667/1305] 10.222.0.83 () {56 vars in 1116 bytes} [Mon Aug 1 15:24:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:24:06.227790 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55670714-6469-45dc-8136-3b0abf3cdd8c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:06.325851 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55670714-6469-45dc-8136-3b0abf3cdd8c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:06.359584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cdadf8-b632-4f93-856e-34f6c87265c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:06.360211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdadf8-b632-4f93-856e-34f6c87265c9 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:06.360495 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdadf8-b632-4f93-856e-34f6c87265c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:06.360770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdadf8-b632-4f93-856e-34f6c87265c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:06.361475 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cdadf8-b632-4f93-856e-34f6c87265c9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:06.362885 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cdadf8-b632-4f93-856e-34f6c87265c9 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:06.382483 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 639/1306] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:24:06 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:16.218387 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=30 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:16.378484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:10.505933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621a8a6-d0d1-4316-bd74-8b567c419bad None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:10.507004 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621a8a6-d0d1-4316-bd74-8b567c419bad None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:10.507511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621a8a6-d0d1-4316-bd74-8b567c419bad None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:10.509809 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 668/1307] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:25:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:25:10.515072 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf15b6f-9ce6-4eaa-a223-7d6a1e3eea4d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:10.515572 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf15b6f-9ce6-4eaa-a223-7d6a1e3eea4d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:10.515701 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf15b6f-9ce6-4eaa-a223-7d6a1e3eea4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:10.602844 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-8bf15b6f-9ce6-4eaa-a223-7d6a1e3eea4d None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:10.604661 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8bf15b6f-9ce6-4eaa-a223-7d6a1e3eea4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:10.838340 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 640/1308] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:25:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:10.883520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da75ff5-b8fc-4699-a4dd-59f277843fcb None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:10.884209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da75ff5-b8fc-4699-a4dd-59f277843fcb None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:10.884711 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da75ff5-b8fc-4699-a4dd-59f277843fcb None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:10.886981 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 669/1309] 10.222.0.83 () {54 vars in 965 bytes} [Mon Aug 1 15:25:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:25:10.895440 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a68ae-7f15-4964-ab82-299245d796c6 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:10.895814 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a68ae-7f15-4964-ab82-299245d796c6 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:10.895996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a68ae-7f15-4964-ab82-299245d796c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:10.985225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d52a68ae-7f15-4964-ab82-299245d796c6 None None] MFA Rules not processed for user `76311c7173b442c0af2c745e3e70a07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:10.986580 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d52a68ae-7f15-4964-ab82-299245d796c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:11.010220 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:11.104036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da75ff5-b8fc-4699-a4dd-59f277843fcb None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:11.105837 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1da75ff5-b8fc-4699-a4dd-59f277843fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:11.109343 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 641/1310] 10.222.0.83 () {58 vars in 1062 bytes} [Mon Aug 1 15:25:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:11.236210 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da75ff5-b8fc-4699-a4dd-59f277843fcb None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:11.317508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888be1d4-0ed2-4ac5-9b6f-f450853679ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:11.318975 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888be1d4-0ed2-4ac5-9b6f-f450853679ee service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:11.319442 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888be1d4-0ed2-4ac5-9b6f-f450853679ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:11.319875 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888be1d4-0ed2-4ac5-9b6f-f450853679ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:11.321797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-888be1d4-0ed2-4ac5-9b6f-f450853679ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:11.388460 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:11.457427 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888be1d4-0ed2-4ac5-9b6f-f450853679ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:11.459413 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888be1d4-0ed2-4ac5-9b6f-f450853679ee service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:11.483178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 670/1311] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:25:13.864836 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692bc41-63c3-493d-a2dd-b2552c42acf6 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:13.865408 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692bc41-63c3-493d-a2dd-b2552c42acf6 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:13.865408 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692bc41-63c3-493d-a2dd-b2552c42acf6 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:13.867081 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 642/1312] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:25:13.872167 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad13a25-2924-4d02-9a80-208fec77bd56 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:13.872564 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad13a25-2924-4d02-9a80-208fec77bd56 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:13.872717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad13a25-2924-4d02-9a80-208fec77bd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:13.965041 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-cad13a25-2924-4d02-9a80-208fec77bd56 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:13.966325 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-cad13a25-2924-4d02-9a80-208fec77bd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:14.078359 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 671/1313] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:14.132463 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692bc41-63c3-493d-a2dd-b2552c42acf6 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:14.134054 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a692bc41-63c3-493d-a2dd-b2552c42acf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:14.239821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692bc41-63c3-493d-a2dd-b2552c42acf6 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:14.313631 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:14.316647 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167110a-ab2f-44ef-9285-b0b9e7ef007d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:14.317490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167110a-ab2f-44ef-9285-b0b9e7ef007d service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:14.317788 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167110a-ab2f-44ef-9285-b0b9e7ef007d service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:14.318874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167110a-ab2f-44ef-9285-b0b9e7ef007d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:14.320776 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c167110a-ab2f-44ef-9285-b0b9e7ef007d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:14.441800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167110a-ab2f-44ef-9285-b0b9e7ef007d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:14.443426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167110a-ab2f-44ef-9285-b0b9e7ef007d service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:14.462318 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 643/1314] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:16.757389 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a72f8e-a98d-4dae-8548-f3dcad8312d6 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:16.757952 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a72f8e-a98d-4dae-8548-f3dcad8312d6 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:16.757952 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a72f8e-a98d-4dae-8548-f3dcad8312d6 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:16.759334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 672/1315] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:25:16.764345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2461948-7190-4b1d-836f-2c0ec36945c7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:16.764761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2461948-7190-4b1d-836f-2c0ec36945c7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:16.764895 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2461948-7190-4b1d-836f-2c0ec36945c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:16.830614 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d2461948-7190-4b1d-836f-2c0ec36945c7 None None] MFA Rules not processed for user `8d5b327a728e436b857d5d3b3675bc2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:16.831839 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d2461948-7190-4b1d-836f-2c0ec36945c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:16.882522 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:16.928641 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 644/1316] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:25:16 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:16.968230 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a72f8e-a98d-4dae-8548-f3dcad8312d6 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:16.969815 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-06a72f8e-a98d-4dae-8548-f3dcad8312d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:16.980424 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:17.074700 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a72f8e-a98d-4dae-8548-f3dcad8312d6 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:17.147605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef425a4-9e91-454f-bd88-4b5bcaee78c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:17.148222 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef425a4-9e91-454f-bd88-4b5bcaee78c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:17.148500 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef425a4-9e91-454f-bd88-4b5bcaee78c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:17.148768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef425a4-9e91-454f-bd88-4b5bcaee78c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:17.150507 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-eef425a4-9e91-454f-bd88-4b5bcaee78c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:17.214003 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:17.263154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef425a4-9e91-454f-bd88-4b5bcaee78c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:17.264582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef425a4-9e91-454f-bd88-4b5bcaee78c8 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:17.283563 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 673/1317] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:16 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:19.944600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fa9cf-6de7-43af-90ae-b9faf5a47bc3 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:19.945211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fa9cf-6de7-43af-90ae-b9faf5a47bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:19.945211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fa9cf-6de7-43af-90ae-b9faf5a47bc3 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:19.946537 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 645/1318] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:25:19.952377 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc89c9-2070-4ae1-9ece-a6dec17b9569 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:19.952855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc89c9-2070-4ae1-9ece-a6dec17b9569 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:19.952959 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc89c9-2070-4ae1-9ece-a6dec17b9569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:20.020067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-94cc89c9-2070-4ae1-9ece-a6dec17b9569 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:20.021446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-94cc89c9-2070-4ae1-9ece-a6dec17b9569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:20.126146 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 674/1319] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:25:19 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:20.181753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b373530-674c-4c11-90ff-d3fa390e87e8 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:20.182066 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b373530-674c-4c11-90ff-d3fa390e87e8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:20.182348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b373530-674c-4c11-90ff-d3fa390e87e8 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:20.183742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 646/1320] 10.222.0.83 () {54 vars in 965 bytes} [Mon Aug 1 15:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:25:20.190564 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bd85a-6029-4c2a-877c-260b4f76389e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:20.190852 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bd85a-6029-4c2a-877c-260b4f76389e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:20.191179 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bd85a-6029-4c2a-877c-260b4f76389e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:20.214471 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:20.272626 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-506bd85a-6029-4c2a-877c-260b4f76389e None None] MFA Rules not processed for user `76311c7173b442c0af2c745e3e70a07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:20.273859 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-506bd85a-6029-4c2a-877c-260b4f76389e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:20.354921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 675/1321] 10.222.0.83 () {58 vars in 1062 bytes} [Mon Aug 1 15:25:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:20.364027 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b373530-674c-4c11-90ff-d3fa390e87e8 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:20.366098 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9b373530-674c-4c11-90ff-d3fa390e87e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:20.470743 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b373530-674c-4c11-90ff-d3fa390e87e8 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:20.553785 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df98478-8bfa-4ee5-bf5f-d0e376663cd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:20.554444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df98478-8bfa-4ee5-bf5f-d0e376663cd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:20.554719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df98478-8bfa-4ee5-bf5f-d0e376663cd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:20.554986 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df98478-8bfa-4ee5-bf5f-d0e376663cd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:20.556629 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2df98478-8bfa-4ee5-bf5f-d0e376663cd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:20.564594 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:20.680581 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df98478-8bfa-4ee5-bf5f-d0e376663cd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:20.682086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df98478-8bfa-4ee5-bf5f-d0e376663cd3 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:20.704225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 647/1322] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:20 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:23.918011 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5740b4d-0a11-4fbd-8e7b-e32cca6d5009 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:23.918593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5740b4d-0a11-4fbd-8e7b-e32cca6d5009 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:23.918593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5740b4d-0a11-4fbd-8e7b-e32cca6d5009 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:23.919926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 676/1323] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:25:23.925214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa5188-663c-4507-ae72-ed3400013cd9 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:23.925643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa5188-663c-4507-ae72-ed3400013cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:23.925768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa5188-663c-4507-ae72-ed3400013cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:23.992716 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f6aa5188-663c-4507-ae72-ed3400013cd9 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:23.993954 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f6aa5188-663c-4507-ae72-ed3400013cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:24.056483 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:24.102207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 648/1324] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:24.140034 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5740b4d-0a11-4fbd-8e7b-e32cca6d5009 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:24.141619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f5740b4d-0a11-4fbd-8e7b-e32cca6d5009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:24.253538 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5740b4d-0a11-4fbd-8e7b-e32cca6d5009 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:24.266073 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:24.332613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7deae3-18f3-4fbb-a031-785be509a955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:24.333241 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7deae3-18f3-4fbb-a031-785be509a955 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:24.333523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7deae3-18f3-4fbb-a031-785be509a955 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:24.333796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7deae3-18f3-4fbb-a031-785be509a955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:24.335468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ae7deae3-18f3-4fbb-a031-785be509a955 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:24.448457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7deae3-18f3-4fbb-a031-785be509a955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:24.449793 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7deae3-18f3-4fbb-a031-785be509a955 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:24.467835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 677/1325] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:26.556097 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aedec-d4d1-40a6-a638-d99a6ad07380 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:26.556624 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aedec-d4d1-40a6-a638-d99a6ad07380 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:26.556624 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aedec-d4d1-40a6-a638-d99a6ad07380 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:26.557909 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 649/1326] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:25:26.563188 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d888f6-cc2f-4f76-9c55-b37defe102ad None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:26.563616 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d888f6-cc2f-4f76-9c55-b37defe102ad None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:26.563953 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d888f6-cc2f-4f76-9c55-b37defe102ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:26.612732 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:26.633590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-48d888f6-cc2f-4f76-9c55-b37defe102ad None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:26.634837 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-48d888f6-cc2f-4f76-9c55-b37defe102ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:26.732422 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 678/1327] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:25:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:26.768526 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18aedec-d4d1-40a6-a638-d99a6ad07380 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:26.770405 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a18aedec-d4d1-40a6-a638-d99a6ad07380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:26.877404 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18aedec-d4d1-40a6-a638-d99a6ad07380 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:26.956034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bf4e2-ea20-47cb-8fc6-3e6bef3d7d40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:26.956670 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bf4e2-ea20-47cb-8fc6-3e6bef3d7d40 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:26.956957 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bf4e2-ea20-47cb-8fc6-3e6bef3d7d40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:26.957235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bf4e2-ea20-47cb-8fc6-3e6bef3d7d40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:26.959425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-605bf4e2-ea20-47cb-8fc6-3e6bef3d7d40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:26.978970 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:27.077730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605bf4e2-ea20-47cb-8fc6-3e6bef3d7d40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:27.079224 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605bf4e2-ea20-47cb-8fc6-3e6bef3d7d40 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:27.097566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 650/1328] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:29.265330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988d1ee-4e6a-41eb-8b87-d9b0c824d653 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:29.265860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988d1ee-4e6a-41eb-8b87-d9b0c824d653 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:29.265860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988d1ee-4e6a-41eb-8b87-d9b0c824d653 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:29.267248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 679/1329] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:25:29.272419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f42e3-d475-46fa-9e6e-dc9035883081 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:29.272805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f42e3-d475-46fa-9e6e-dc9035883081 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:29.272967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f42e3-d475-46fa-9e6e-dc9035883081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:29.338917 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f93f42e3-d475-46fa-9e6e-dc9035883081 None None] MFA Rules not processed for user `8d5b327a728e436b857d5d3b3675bc2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:29.340033 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f93f42e3-d475-46fa-9e6e-dc9035883081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:29.408302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:29.444653 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 651/1330] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:25:29 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:29.483187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988d1ee-4e6a-41eb-8b87-d9b0c824d653 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:29.484810 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1988d1ee-4e6a-41eb-8b87-d9b0c824d653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:29.597595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988d1ee-4e6a-41eb-8b87-d9b0c824d653 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:29.618618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:29.673242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea364d9-a32e-4b21-b4ac-07040cd2e5b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:29.674025 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea364d9-a32e-4b21-b4ac-07040cd2e5b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:29.674304 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea364d9-a32e-4b21-b4ac-07040cd2e5b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:29.674571 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea364d9-a32e-4b21-b4ac-07040cd2e5b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:29.676321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-eea364d9-a32e-4b21-b4ac-07040cd2e5b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:29.791824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea364d9-a32e-4b21-b4ac-07040cd2e5b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:29.793263 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea364d9-a32e-4b21-b4ac-07040cd2e5b3 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:29.814689 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 680/1331] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:29 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:32.110618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552fb47-c019-498b-85a7-3c1c597134c0 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:32.111138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552fb47-c019-498b-85a7-3c1c597134c0 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:32.111138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552fb47-c019-498b-85a7-3c1c597134c0 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:32.112510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 652/1332] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:25:32.117606 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfcb2a-b5cb-4c1d-9d61-c51ff805dfaa None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:32.118016 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfcb2a-b5cb-4c1d-9d61-c51ff805dfaa None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:32.118317 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfcb2a-b5cb-4c1d-9d61-c51ff805dfaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:32.185524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:32.188817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-0cbfcb2a-b5cb-4c1d-9d61-c51ff805dfaa None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:32.191011 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0cbfcb2a-b5cb-4c1d-9d61-c51ff805dfaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:32.291898 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 681/1333] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:32.328404 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552fb47-c019-498b-85a7-3c1c597134c0 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:32.330001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a552fb47-c019-498b-85a7-3c1c597134c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:32.441501 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552fb47-c019-498b-85a7-3c1c597134c0 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:32.517286 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b73ad5-a580-4f42-bccd-01a4fa7f747e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:32.518323 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b73ad5-a580-4f42-bccd-01a4fa7f747e service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:32.518672 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b73ad5-a580-4f42-bccd-01a4fa7f747e service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:32.519032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b73ad5-a580-4f42-bccd-01a4fa7f747e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:32.521507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-55b73ad5-a580-4f42-bccd-01a4fa7f747e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:32.550387 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:32.636479 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b73ad5-a580-4f42-bccd-01a4fa7f747e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:32.638041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b73ad5-a580-4f42-bccd-01a4fa7f747e service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:32.659343 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 653/1334] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:34.760353 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8cab3-b05f-4248-821e-e5b6ad17e233 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:34.761002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8cab3-b05f-4248-821e-e5b6ad17e233 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:34.761002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8cab3-b05f-4248-821e-e5b6ad17e233 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:34.762407 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 682/1335] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:25:34.767639 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb0dcd-0444-4e8b-a927-2e81e8a8d82b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:34.768015 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb0dcd-0444-4e8b-a927-2e81e8a8d82b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:34.768200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb0dcd-0444-4e8b-a927-2e81e8a8d82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:34.835626 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-85cb0dcd-0444-4e8b-a927-2e81e8a8d82b None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:34.837027 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-85cb0dcd-0444-4e8b-a927-2e81e8a8d82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:34.918942 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:34.946471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 654/1336] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:34.987238 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8cab3-b05f-4248-821e-e5b6ad17e233 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:34.988766 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-51e8cab3-b05f-4248-821e-e5b6ad17e233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:35.094076 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8cab3-b05f-4248-821e-e5b6ad17e233 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:35.122564 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:35.167920 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7ff9b-f04c-4aad-9e5d-5013dbc1be3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:35.168544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7ff9b-f04c-4aad-9e5d-5013dbc1be3e service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:35.168821 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7ff9b-f04c-4aad-9e5d-5013dbc1be3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:35.169089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7ff9b-f04c-4aad-9e5d-5013dbc1be3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:35.170773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9be7ff9b-f04c-4aad-9e5d-5013dbc1be3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:35.283794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7ff9b-f04c-4aad-9e5d-5013dbc1be3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:35.287182 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7ff9b-f04c-4aad-9e5d-5013dbc1be3e service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:35.309353 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 683/1337] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:37.774764 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ca457-d0fa-439f-9d7f-39b666cc21dd None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:37.775297 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ca457-d0fa-439f-9d7f-39b666cc21dd None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:37.775418 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ca457-d0fa-439f-9d7f-39b666cc21dd None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:37.776685 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 655/1338] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:25:37.781945 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbd9c1-94db-4c90-aa38-3ccebd6bfb5a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:37.782379 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbd9c1-94db-4c90-aa38-3ccebd6bfb5a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:37.782511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbd9c1-94db-4c90-aa38-3ccebd6bfb5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:37.850587 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-ebdbd9c1-94db-4c90-aa38-3ccebd6bfb5a None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:37.851735 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ebdbd9c1-94db-4c90-aa38-3ccebd6bfb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:37.860480 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:37.950485 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 684/1339] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:25:37 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:37.987212 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ca457-d0fa-439f-9d7f-39b666cc21dd None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:37.988747 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d40ca457-d0fa-439f-9d7f-39b666cc21dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:38.094996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ca457-d0fa-439f-9d7f-39b666cc21dd None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:38.169435 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc3796-79a8-496c-b08e-11527da2652d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:38.170168 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc3796-79a8-496c-b08e-11527da2652d service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:38.170465 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc3796-79a8-496c-b08e-11527da2652d service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:38.170741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc3796-79a8-496c-b08e-11527da2652d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:38.172384 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-02bc3796-79a8-496c-b08e-11527da2652d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:38.213771 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:38.286160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc3796-79a8-496c-b08e-11527da2652d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:38.287599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc3796-79a8-496c-b08e-11527da2652d service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:38.306217 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 656/1340] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:37 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:42.320616 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbe4a5-5448-45f9-8462-b5c2c9e8c1f5 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:42.321147 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbe4a5-5448-45f9-8462-b5c2c9e8c1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:42.321147 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbe4a5-5448-45f9-8462-b5c2c9e8c1f5 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:42.322486 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 685/1341] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:25:42.327911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde4871-33d3-487b-8df2-c5d5267deac6 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:42.328336 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde4871-33d3-487b-8df2-c5d5267deac6 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:42.328498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde4871-33d3-487b-8df2-c5d5267deac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:42.396778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3cde4871-33d3-487b-8df2-c5d5267deac6 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:42.398022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3cde4871-33d3-487b-8df2-c5d5267deac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:42.489959 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:42.501811 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 657/1342] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:42.543661 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafbe4a5-5448-45f9-8462-b5c2c9e8c1f5 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:42.545184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-dafbe4a5-5448-45f9-8462-b5c2c9e8c1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:42.658457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafbe4a5-5448-45f9-8462-b5c2c9e8c1f5 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:42.697251 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:42.735728 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf4f25-fe8f-4f36-8c17-414e629b1bd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:42.736350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf4f25-fe8f-4f36-8c17-414e629b1bd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:42.736631 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf4f25-fe8f-4f36-8c17-414e629b1bd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:42.736902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf4f25-fe8f-4f36-8c17-414e629b1bd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:42.738577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fedf4f25-fe8f-4f36-8c17-414e629b1bd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:42.852178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf4f25-fe8f-4f36-8c17-414e629b1bd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:42.853607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf4f25-fe8f-4f36-8c17-414e629b1bd0 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:42.872303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 686/1343] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:45.682932 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bed758-2275-4499-b315-b31344f80294 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:45.683447 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bed758-2275-4499-b315-b31344f80294 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:45.683540 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bed758-2275-4499-b315-b31344f80294 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:45.684878 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 658/1344] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:25:45.690690 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db069388-bcc8-442a-baa3-67feace0aa9f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:45.691246 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db069388-bcc8-442a-baa3-67feace0aa9f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:45.691246 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db069388-bcc8-442a-baa3-67feace0aa9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:45.761681 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-db069388-bcc8-442a-baa3-67feace0aa9f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:45.762932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-db069388-bcc8-442a-baa3-67feace0aa9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:45.781227 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:45.861453 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 687/1345] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:25:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:45.899807 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bed758-2275-4499-b315-b31344f80294 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:45.901361 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-21bed758-2275-4499-b315-b31344f80294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:46.012225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bed758-2275-4499-b315-b31344f80294 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:46.091131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c89a5-8876-420b-9611-e7ea199d8f05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:46.091763 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c89a5-8876-420b-9611-e7ea199d8f05 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:46.092048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c89a5-8876-420b-9611-e7ea199d8f05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:46.092324 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c89a5-8876-420b-9611-e7ea199d8f05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:46.093978 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7f9c89a5-8876-420b-9611-e7ea199d8f05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:46.137433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:46.207803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c89a5-8876-420b-9611-e7ea199d8f05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:46.209804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c89a5-8876-420b-9611-e7ea199d8f05 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:46.229290 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 659/1346] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:45 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:50.296092 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfcf703-3326-429f-b18a-d2cc04b28956 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.296665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfcf703-3326-429f-b18a-d2cc04b28956 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.296665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfcf703-3326-429f-b18a-d2cc04b28956 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.298029 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 688/1347] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:25:50.304322 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc43141-a02e-4a2a-9d1f-2fde2a2b41ea None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.304853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc43141-a02e-4a2a-9d1f-2fde2a2b41ea None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.304965 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc43141-a02e-4a2a-9d1f-2fde2a2b41ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.402919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-5fc43141-a02e-4a2a-9d1f-2fde2a2b41ea None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:50.404187 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5fc43141-a02e-4a2a-9d1f-2fde2a2b41ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:50.525776 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:50.539235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 660/1348] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:50.585580 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfcf703-3326-429f-b18a-d2cc04b28956 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:50.587201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2cfcf703-3326-429f-b18a-d2cc04b28956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:50.716732 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfcf703-3326-429f-b18a-d2cc04b28956 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:50.772442 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:50.807692 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6977a1-96b3-486f-9427-b381ddee900f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:50.808463 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6977a1-96b3-486f-9427-b381ddee900f service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.808750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6977a1-96b3-486f-9427-b381ddee900f service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.809024 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6977a1-96b3-486f-9427-b381ddee900f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.811803 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7a6977a1-96b3-486f-9427-b381ddee900f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:50.939008 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6977a1-96b3-486f-9427-b381ddee900f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:50.940346 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6977a1-96b3-486f-9427-b381ddee900f service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:50.960905 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 689/1349] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:55.852440 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df19ce-df3c-4e8e-9a8b-62030a867b0c None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:55.852951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df19ce-df3c-4e8e-9a8b-62030a867b0c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:55.853053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df19ce-df3c-4e8e-9a8b-62030a867b0c None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:55.854935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 661/1350] 10.222.0.83 () {54 vars in 921 bytes} [Mon Aug 1 15:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:25:55.861695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf170a-f21b-4bf8-952e-00f54442565d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:55.863336 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf170a-f21b-4bf8-952e-00f54442565d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:55.863850 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf170a-f21b-4bf8-952e-00f54442565d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:55.991703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-ecaf170a-f21b-4bf8-952e-00f54442565d None None] MFA Rules not processed for user `ef57a9394b114eb3ab499133b5d40781`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:55.992882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ecaf170a-f21b-4bf8-952e-00f54442565d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:56.015935 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:56.121682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 690/1351] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Aug 1 15:25:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:00.459445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456af52-4698-4b1e-985a-1873a6bd83b1 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:00.460023 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456af52-4698-4b1e-985a-1873a6bd83b1 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:00.460107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456af52-4698-4b1e-985a-1873a6bd83b1 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:00.462153 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 662/1352] 10.222.0.83 () {54 vars in 921 bytes} [Mon Aug 1 15:26:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:00.468212 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6285d-43fa-4bdb-8b01-c98c8f2511e7 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:00.468681 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6285d-43fa-4bdb-8b01-c98c8f2511e7 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:00.468783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6285d-43fa-4bdb-8b01-c98c8f2511e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:00.552828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-56b6285d-43fa-4bdb-8b01-c98c8f2511e7 None None] MFA Rules not processed for user `ef57a9394b114eb3ab499133b5d40781`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:00.554302 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-56b6285d-43fa-4bdb-8b01-c98c8f2511e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:00.656677 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:00.696377 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 691/1353] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Aug 1 15:26:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:01.905100 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac60dd4-0891-4818-81b8-834187b6f838 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:01.905623 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac60dd4-0891-4818-81b8-834187b6f838 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:01.905623 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac60dd4-0891-4818-81b8-834187b6f838 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:01.907059 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 663/1354] 10.222.0.83 () {54 vars in 921 bytes} [Mon Aug 1 15:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:01.912455 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824c0cf-441f-4ca1-83a5-b2a066445504 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:01.912882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824c0cf-441f-4ca1-83a5-b2a066445504 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:01.913022 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824c0cf-441f-4ca1-83a5-b2a066445504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:01.991032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f824c0cf-441f-4ca1-83a5-b2a066445504 None None] MFA Rules not processed for user `ef57a9394b114eb3ab499133b5d40781`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:01.992225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f824c0cf-441f-4ca1-83a5-b2a066445504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:02.103090 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 692/1355] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Aug 1 15:26:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:05.647318 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667828a4-a225-4b68-a77e-4ca3e59174c8 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:05.648174 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667828a4-a225-4b68-a77e-4ca3e59174c8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:05.648631 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667828a4-a225-4b68-a77e-4ca3e59174c8 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:05.650550 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 664/1356] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:26:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:05.656109 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f3d88-769c-4005-9cd8-fba505f820ac None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:05.656964 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f3d88-769c-4005-9cd8-fba505f820ac None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:05.657398 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f3d88-769c-4005-9cd8-fba505f820ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:05.704977 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:05.743784 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-7a2f3d88-769c-4005-9cd8-fba505f820ac None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:05.745005 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7a2f3d88-769c-4005-9cd8-fba505f820ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:05.867538 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 693/1357] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:05.929998 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a15fd-c8ec-4691-98a3-921fa018d78e None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:05.930557 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a15fd-c8ec-4691-98a3-921fa018d78e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:05.930996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a15fd-c8ec-4691-98a3-921fa018d78e None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:05.932870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 665/1358] 10.222.0.83 () {54 vars in 962 bytes} [Mon Aug 1 15:26:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:05.941576 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35dbf9-5632-4a68-8878-37b87f1111bd None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:05.942603 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35dbf9-5632-4a68-8878-37b87f1111bd None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:05.943038 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35dbf9-5632-4a68-8878-37b87f1111bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:06.028275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-7b35dbf9-5632-4a68-8878-37b87f1111bd None None] MFA Rules not processed for user `25dc41ffc41c47fdbeec06110d04610c`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:06.029491 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7b35dbf9-5632-4a68-8878-37b87f1111bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:06.092644 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:06.166274 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 694/1359] 10.222.0.83 () {58 vars in 1059 bytes} [Mon Aug 1 15:26:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:26:06.173752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a15fd-c8ec-4691-98a3-921fa018d78e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:06.175462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ba0a15fd-c8ec-4691-98a3-921fa018d78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:06.340500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a15fd-c8ec-4691-98a3-921fa018d78e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:06.420118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:06.465545 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18560ca-8528-4fc5-b21c-81388820b540 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:06.466315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18560ca-8528-4fc5-b21c-81388820b540 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:06.466702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18560ca-8528-4fc5-b21c-81388820b540 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:06.466958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18560ca-8528-4fc5-b21c-81388820b540 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:06.468714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e18560ca-8528-4fc5-b21c-81388820b540 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:06.568382 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18560ca-8528-4fc5-b21c-81388820b540 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:06.570554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18560ca-8528-4fc5-b21c-81388820b540 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:06.591718 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 666/1360] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:08.837676 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def0424-227b-4043-9f62-d4ed49b131b9 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:08.838260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def0424-227b-4043-9f62-d4ed49b131b9 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:08.838345 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def0424-227b-4043-9f62-d4ed49b131b9 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:08.839856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 695/1361] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:08.845369 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6de0c-f2cf-4f0d-9d28-6f4fad37a229 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:08.845958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6de0c-f2cf-4f0d-9d28-6f4fad37a229 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:08.845958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6de0c-f2cf-4f0d-9d28-6f4fad37a229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:08.920215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c0e6de0c-f2cf-4f0d-9d28-6f4fad37a229 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:08.921410 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c0e6de0c-f2cf-4f0d-9d28-6f4fad37a229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:08.964057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:09.028917 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 667/1362] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:09.076418 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def0424-227b-4043-9f62-d4ed49b131b9 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:09.079431 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6def0424-227b-4043-9f62-d4ed49b131b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:09.254338 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def0424-227b-4043-9f62-d4ed49b131b9 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:09.264470 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:09.443941 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe580f-857d-4e86-aad7-9f63e5766432 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:09.445595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe580f-857d-4e86-aad7-9f63e5766432 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:09.445595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe580f-857d-4e86-aad7-9f63e5766432 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:09.445595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe580f-857d-4e86-aad7-9f63e5766432 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:09.447370 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-45fe580f-857d-4e86-aad7-9f63e5766432 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:09.540524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe580f-857d-4e86-aad7-9f63e5766432 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:09.542175 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe580f-857d-4e86-aad7-9f63e5766432 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:09.561264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 696/1363] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:12.011017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c8dbc-a712-44f3-840c-e2d41ef01b56 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:12.011017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c8dbc-a712-44f3-840c-e2d41ef01b56 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:12.011017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c8dbc-a712-44f3-840c-e2d41ef01b56 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:12.012482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 668/1364] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:12.018234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ba421-59d0-4190-a3af-48b9abebe615 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:12.019086 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ba421-59d0-4190-a3af-48b9abebe615 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:12.019624 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ba421-59d0-4190-a3af-48b9abebe615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:12.036996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:12.096540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-379ba421-59d0-4190-a3af-48b9abebe615 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:12.097775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-379ba421-59d0-4190-a3af-48b9abebe615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:12.207234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 697/1365] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:12.253041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f51b7-427e-49d1-baf6-5a4ed20507d9 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:12.253407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f51b7-427e-49d1-baf6-5a4ed20507d9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:12.253758 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f51b7-427e-49d1-baf6-5a4ed20507d9 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:12.255488 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 669/1366] 10.222.0.83 () {54 vars in 962 bytes} [Mon Aug 1 15:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:12.261353 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6858d5b-4349-4eb6-ad47-4994ed6c65ae None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:12.261617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6858d5b-4349-4eb6-ad47-4994ed6c65ae None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:12.261858 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6858d5b-4349-4eb6-ad47-4994ed6c65ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:12.346327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f6858d5b-4349-4eb6-ad47-4994ed6c65ae None None] MFA Rules not processed for user `25dc41ffc41c47fdbeec06110d04610c`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:12.348143 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f6858d5b-4349-4eb6-ad47-4994ed6c65ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:12.386623 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:12.479471 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 698/1367] 10.222.0.83 () {58 vars in 1059 bytes} [Mon Aug 1 15:26:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:12.487098 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f51b7-427e-49d1-baf6-5a4ed20507d9 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:12.489305 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a02f51b7-427e-49d1-baf6-5a4ed20507d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:12.659576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f51b7-427e-49d1-baf6-5a4ed20507d9 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:12.683117 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:12.791063 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cb635-b2c6-4940-933f-03e1ba143580 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:12.792089 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cb635-b2c6-4940-933f-03e1ba143580 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:12.792608 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cb635-b2c6-4940-933f-03e1ba143580 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:12.793110 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cb635-b2c6-4940-933f-03e1ba143580 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:12.795192 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e28cb635-b2c6-4940-933f-03e1ba143580 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:12.896508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cb635-b2c6-4940-933f-03e1ba143580 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:12.898608 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cb635-b2c6-4940-933f-03e1ba143580 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:12.920696 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 670/1368] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:13.891394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be334b-11e6-4eac-8924-e07e019d75ce None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:13.891981 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be334b-11e6-4eac-8924-e07e019d75ce None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:13.891981 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be334b-11e6-4eac-8924-e07e019d75ce None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:13.894753 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 699/1369] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:13.899313 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d211229-2b22-4969-821d-42cf8438b651 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:13.899825 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d211229-2b22-4969-821d-42cf8438b651 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:13.899922 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d211229-2b22-4969-821d-42cf8438b651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:13.935499 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:13.975526 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-5d211229-2b22-4969-821d-42cf8438b651 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:13.976763 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5d211229-2b22-4969-821d-42cf8438b651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:14.091979 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 671/1370] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:14.444520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df031e2-33b5-4581-94a6-3883dfcc2891 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:14.444829 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df031e2-33b5-4581-94a6-3883dfcc2891 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:14.445104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df031e2-33b5-4581-94a6-3883dfcc2891 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:14.446913 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 700/1371] 10.222.0.83 () {54 vars in 964 bytes} [Mon Aug 1 15:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:14.452488 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c1384-02a9-40ed-92c5-144367f70533 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:14.452954 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c1384-02a9-40ed-92c5-144367f70533 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:14.453231 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c1384-02a9-40ed-92c5-144367f70533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:14.538041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-de8c1384-02a9-40ed-92c5-144367f70533 None None] MFA Rules not processed for user `f54b570990794801a5e45918845569b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:14.539268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-de8c1384-02a9-40ed-92c5-144367f70533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:14.590509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:14.627373 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 672/1372] 10.222.0.83 () {58 vars in 1061 bytes} [Mon Aug 1 15:26:14 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:14.635212 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df031e2-33b5-4581-94a6-3883dfcc2891 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:14.636724 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7df031e2-33b5-4581-94a6-3883dfcc2891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:14.761216 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df031e2-33b5-4581-94a6-3883dfcc2891 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:14.806634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:14.844863 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c6e80-5a2f-4b37-a973-5305fa20bb12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:14.845491 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c6e80-5a2f-4b37-a973-5305fa20bb12 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:14.845773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c6e80-5a2f-4b37-a973-5305fa20bb12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:14.846085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c6e80-5a2f-4b37-a973-5305fa20bb12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:14.848303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7e6c6e80-5a2f-4b37-a973-5305fa20bb12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:14.967963 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c6e80-5a2f-4b37-a973-5305fa20bb12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:14.970532 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c6e80-5a2f-4b37-a973-5305fa20bb12 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:14.997095 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 701/1373] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:15.058935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958345c-6170-4b8a-9830-f92b8216e12e None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:15.059344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958345c-6170-4b8a-9830-f92b8216e12e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:15.059627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958345c-6170-4b8a-9830-f92b8216e12e None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:15.061408 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 673/1374] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:15.067058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78de0f6-d130-477c-8018-efb11c6e91f7 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:15.067497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78de0f6-d130-477c-8018-efb11c6e91f7 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:15.067847 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78de0f6-d130-477c-8018-efb11c6e91f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:15.149536 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a78de0f6-d130-477c-8018-efb11c6e91f7 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:15.151089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a78de0f6-d130-477c-8018-efb11c6e91f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:15.176975 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:15.262204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 702/1375] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:26:15.300952 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958345c-6170-4b8a-9830-f92b8216e12e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:15.302417 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6958345c-6170-4b8a-9830-f92b8216e12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:15.472382 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958345c-6170-4b8a-9830-f92b8216e12e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:15.525288 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:15.591408 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8001d-bf2b-4b76-9461-b3462c2cf9bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:15.592043 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8001d-bf2b-4b76-9461-b3462c2cf9bb service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:15.592327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8001d-bf2b-4b76-9461-b3462c2cf9bb service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:15.592602 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8001d-bf2b-4b76-9461-b3462c2cf9bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:15.594278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-09a8001d-bf2b-4b76-9461-b3462c2cf9bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:15.701410 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8001d-bf2b-4b76-9461-b3462c2cf9bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:15.703494 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8001d-bf2b-4b76-9461-b3462c2cf9bb service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:15.728669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 674/1376] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:17.816710 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afebbae-d696-4162-aeb4-5be832e7a677 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:17.817231 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afebbae-d696-4162-aeb4-5be832e7a677 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:17.817231 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afebbae-d696-4162-aeb4-5be832e7a677 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:17.818556 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 703/1377] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:17.823817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf1eea-6433-4fb4-9a1e-601c969a94c4 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:17.824268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf1eea-6433-4fb4-9a1e-601c969a94c4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:17.824496 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf1eea-6433-4fb4-9a1e-601c969a94c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:17.891604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-cdcf1eea-6433-4fb4-9a1e-601c969a94c4 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:17.892832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-cdcf1eea-6433-4fb4-9a1e-601c969a94c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:17.902183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:17.999548 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 675/1378] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:18.046062 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afebbae-d696-4162-aeb4-5be832e7a677 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:18.047588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1afebbae-d696-4162-aeb4-5be832e7a677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:18.223311 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afebbae-d696-4162-aeb4-5be832e7a677 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:18.226781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:18.342742 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7b2ca-8af7-4724-94fa-95bf964ee175 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:18.343370 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7b2ca-8af7-4724-94fa-95bf964ee175 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:18.343647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7b2ca-8af7-4724-94fa-95bf964ee175 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:18.343912 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7b2ca-8af7-4724-94fa-95bf964ee175 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:18.345536 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6ef7b2ca-8af7-4724-94fa-95bf964ee175 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:18.452038 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7b2ca-8af7-4724-94fa-95bf964ee175 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:18.454176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7b2ca-8af7-4724-94fa-95bf964ee175 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:18.475441 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 704/1379] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:20.587265 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d3c3d-8d85-4f4b-b60c-6c47c5ba289f None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:20.587884 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d3c3d-8d85-4f4b-b60c-6c47c5ba289f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:20.588150 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d3c3d-8d85-4f4b-b60c-6c47c5ba289f None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:20.590032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 676/1380] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:26:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:20.595260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7bc88-3f4d-487a-b109-d916b22f6489 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:20.595856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7bc88-3f4d-487a-b109-d916b22f6489 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:20.595856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7bc88-3f4d-487a-b109-d916b22f6489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:20.608153 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:20.677061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-7dc7bc88-3f4d-487a-b109-d916b22f6489 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:20.678281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7dc7bc88-3f4d-487a-b109-d916b22f6489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:20.794373 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 705/1381] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:20.849882 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d3c3d-8d85-4f4b-b60c-6c47c5ba289f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:20.851562 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d55d3c3d-8d85-4f4b-b60c-6c47c5ba289f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:21.038731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:21.054589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d3c3d-8d85-4f4b-b60c-6c47c5ba289f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:21.189303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50656aa5-26f9-42ac-b0e8-c7b1a5aaec65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:21.189952 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50656aa5-26f9-42ac-b0e8-c7b1a5aaec65 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:21.190245 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50656aa5-26f9-42ac-b0e8-c7b1a5aaec65 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:21.190530 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50656aa5-26f9-42ac-b0e8-c7b1a5aaec65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:21.192860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-50656aa5-26f9-42ac-b0e8-c7b1a5aaec65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:21.310089 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50656aa5-26f9-42ac-b0e8-c7b1a5aaec65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:21.311791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50656aa5-26f9-42ac-b0e8-c7b1a5aaec65 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:21.323975 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:21.335417 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 677/1382] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:23.476293 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bfc7d-4735-47e5-9495-4145863f8c03 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:23.476849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bfc7d-4735-47e5-9495-4145863f8c03 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:23.477098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bfc7d-4735-47e5-9495-4145863f8c03 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:23.478680 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 706/1383] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:23.485132 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fba48-27df-4c1f-9a81-2a6f3ba179a9 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:23.485724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fba48-27df-4c1f-9a81-2a6f3ba179a9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:23.485724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fba48-27df-4c1f-9a81-2a6f3ba179a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:23.557374 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6b1fba48-27df-4c1f-9a81-2a6f3ba179a9 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:23.558738 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6b1fba48-27df-4c1f-9a81-2a6f3ba179a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:23.661701 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 678/1384] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:23.704124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bfc7d-4735-47e5-9495-4145863f8c03 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:23.705859 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fa4bfc7d-4735-47e5-9495-4145863f8c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:23.796140 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:23.861955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bfc7d-4735-47e5-9495-4145863f8c03 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:23.986754 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40b9a5-8470-467d-b888-cf7f65f92572 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:23.987330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40b9a5-8470-467d-b888-cf7f65f92572 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:23.987617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40b9a5-8470-467d-b888-cf7f65f92572 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:23.988387 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40b9a5-8470-467d-b888-cf7f65f92572 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:23.990075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2a40b9a5-8470-467d-b888-cf7f65f92572 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:24.043379 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:24.089574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40b9a5-8470-467d-b888-cf7f65f92572 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:24.091889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40b9a5-8470-467d-b888-cf7f65f92572 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:24.115811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 707/1385] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:26.242874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f5ba4-ec16-4c67-a633-c4a38e2796ec None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:26.243851 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f5ba4-ec16-4c67-a633-c4a38e2796ec None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:26.244614 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f5ba4-ec16-4c67-a633-c4a38e2796ec None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:26.246449 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 679/1386] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:26:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:26.254071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8838c28-56cd-44d1-93ae-7a2b3674b882 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:26.254561 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8838c28-56cd-44d1-93ae-7a2b3674b882 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:26.254838 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8838c28-56cd-44d1-93ae-7a2b3674b882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:26.338940 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a8838c28-56cd-44d1-93ae-7a2b3674b882 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:26.340449 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a8838c28-56cd-44d1-93ae-7a2b3674b882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:26.451882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 708/1387] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:26.508834 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f5ba4-ec16-4c67-a633-c4a38e2796ec None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:26.510448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1c4f5ba4-ec16-4c67-a633-c4a38e2796ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:26.594096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:26.670440 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f5ba4-ec16-4c67-a633-c4a38e2796ec None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:26.791268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519e600-8cd4-42ea-94b4-e3625b3124e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:26.792131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519e600-8cd4-42ea-94b4-e3625b3124e6 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:26.792495 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519e600-8cd4-42ea-94b4-e3625b3124e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:26.792862 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519e600-8cd4-42ea-94b4-e3625b3124e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:26.795017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8519e600-8cd4-42ea-94b4-e3625b3124e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:26.844483 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:26.913635 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519e600-8cd4-42ea-94b4-e3625b3124e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:26.915334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519e600-8cd4-42ea-94b4-e3625b3124e6 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:26.934299 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 680/1388] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:29.024131 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518d29f-e5a9-4e11-8632-fbd0b2bca8f9 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:29.024712 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518d29f-e5a9-4e11-8632-fbd0b2bca8f9 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:29.024712 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518d29f-e5a9-4e11-8632-fbd0b2bca8f9 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:29.026046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 709/1389] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:29.031140 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc162e-d920-4b72-8b3c-c957e8a238b4 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:29.031577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc162e-d920-4b72-8b3c-c957e8a238b4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:29.031683 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc162e-d920-4b72-8b3c-c957e8a238b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:29.104401 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-60bc162e-d920-4b72-8b3c-c957e8a238b4 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:29.105629 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-60bc162e-d920-4b72-8b3c-c957e8a238b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:29.205403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 681/1390] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:29.249489 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518d29f-e5a9-4e11-8632-fbd0b2bca8f9 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:29.251145 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d518d29f-e5a9-4e11-8632-fbd0b2bca8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:29.282035 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:29.400568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518d29f-e5a9-4e11-8632-fbd0b2bca8f9 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:29.504202 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:29.516192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8bacb4-fe13-4216-b0fd-8b568dabe56a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:29.516812 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8bacb4-fe13-4216-b0fd-8b568dabe56a service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:29.517089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8bacb4-fe13-4216-b0fd-8b568dabe56a service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:29.517356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8bacb4-fe13-4216-b0fd-8b568dabe56a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:29.519023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-de8bacb4-fe13-4216-b0fd-8b568dabe56a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:29.608839 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8bacb4-fe13-4216-b0fd-8b568dabe56a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:29.610619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8bacb4-fe13-4216-b0fd-8b568dabe56a service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:29.635753 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 710/1391] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:30.571622 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4f70a-276f-489b-a1e9-d0e8884755d9 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:30.572192 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4f70a-276f-489b-a1e9-d0e8884755d9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:30.572192 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4f70a-276f-489b-a1e9-d0e8884755d9 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:30.573517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 682/1392] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:26:30.579166 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8758f7-98ce-4f42-aad3-637b212af3cf None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:30.579690 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8758f7-98ce-4f42-aad3-637b212af3cf None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:30.579807 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8758f7-98ce-4f42-aad3-637b212af3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:30.650179 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-aa8758f7-98ce-4f42-aad3-637b212af3cf None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:30.651394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-aa8758f7-98ce-4f42-aad3-637b212af3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:30.711582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:30.753334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 711/1393] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:26:30.819904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321135a-0b5b-470e-8b34-86798a164437 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:30.820199 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321135a-0b5b-470e-8b34-86798a164437 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:30.820441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321135a-0b5b-470e-8b34-86798a164437 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:30.821843 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 683/1394] 10.222.0.83 () {54 vars in 966 bytes} [Mon Aug 1 15:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:26:30.827600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca383a2-2c10-429f-b1e5-a021bd84f6e4 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:30.827912 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca383a2-2c10-429f-b1e5-a021bd84f6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:30.828158 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca383a2-2c10-429f-b1e5-a021bd84f6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:30.912510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-bca383a2-2c10-429f-b1e5-a021bd84f6e4 None None] MFA Rules not processed for user `69d216a568b546589e854c2b4476efe6`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:30.913724 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-bca383a2-2c10-429f-b1e5-a021bd84f6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:30.991648 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 712/1395] 10.222.0.83 () {58 vars in 1063 bytes} [Mon Aug 1 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:30.999116 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321135a-0b5b-470e-8b34-86798a164437 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:31.000625 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1321135a-0b5b-470e-8b34-86798a164437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:31.014640 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:31.121813 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321135a-0b5b-470e-8b34-86798a164437 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:31.198997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29a038-f57c-4ea5-8c7f-99c1f869b9e0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69d216a568b546589e854c2b4476efe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:31.199626 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29a038-f57c-4ea5-8c7f-99c1f869b9e0 service glance] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:31.199911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29a038-f57c-4ea5-8c7f-99c1f869b9e0 service glance] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:31.200182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29a038-f57c-4ea5-8c7f-99c1f869b9e0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:31.202099 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7f29a038-f57c-4ea5-8c7f-99c1f869b9e0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:31.274157 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:31.317962 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f29a038-f57c-4ea5-8c7f-99c1f869b9e0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:31.319403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f29a038-f57c-4ea5-8c7f-99c1f869b9e0 service glance] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:31.338420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 684/1396] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:31.744824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f233c-aed9-4fd2-8e91-3a945c2db801 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:31.745390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f233c-aed9-4fd2-8e91-3a945c2db801 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:31.745593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f233c-aed9-4fd2-8e91-3a945c2db801 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:31.747073 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 713/1397] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:31.752594 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d47b6-17cc-46fc-9e86-bcb7b9fb78f7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:31.753002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d47b6-17cc-46fc-9e86-bcb7b9fb78f7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:31.753155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d47b6-17cc-46fc-9e86-bcb7b9fb78f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:31.824292 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-1d2d47b6-17cc-46fc-9e86-bcb7b9fb78f7 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:31.825606 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1d2d47b6-17cc-46fc-9e86-bcb7b9fb78f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:31.929573 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 685/1398] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:31.976178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f233c-aed9-4fd2-8e91-3a945c2db801 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:31.977763 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a89f233c-aed9-4fd2-8e91-3a945c2db801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:32.051920 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:32.101835 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386d5c1-8aa5-4100-bfa0-78a4930d48a4 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:32.102379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386d5c1-8aa5-4100-bfa0-78a4930d48a4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:32.102498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386d5c1-8aa5-4100-bfa0-78a4930d48a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:32.130398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:32.144287 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f233c-aed9-4fd2-8e91-3a945c2db801 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:32.170505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-8386d5c1-8aa5-4100-bfa0-78a4930d48a4 None None] MFA Rules not processed for user `f36c6987ef774a1684c55f885e2a4f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:32.171736 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8386d5c1-8aa5-4100-bfa0-78a4930d48a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:32.262237 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dceb1a8-57a0-475c-87b2-4ef5fe896621 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:32.262776 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dceb1a8-57a0-475c-87b2-4ef5fe896621 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:32.263057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dceb1a8-57a0-475c-87b2-4ef5fe896621 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:32.263327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dceb1a8-57a0-475c-87b2-4ef5fe896621 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:32.264456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 686/1399] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Aug 1 15:26:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:26:32.264983 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2dceb1a8-57a0-475c-87b2-4ef5fe896621 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:32.288793 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:32.312090 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc692af-53ce-4882-9b29-385648b9ac21 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:32.312377 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc692af-53ce-4882-9b29-385648b9ac21 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:32.312646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc692af-53ce-4882-9b29-385648b9ac21 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:32.314033 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 687/1400] 10.222.0.83 () {54 vars in 943 bytes} [Mon Aug 1 15:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:26:32.320471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05386e5e-47a8-43f3-900e-9f29d216a86d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:32.320786 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05386e5e-47a8-43f3-900e-9f29d216a86d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:32.321026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05386e5e-47a8-43f3-900e-9f29d216a86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:32.362938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dceb1a8-57a0-475c-87b2-4ef5fe896621 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:32.364685 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dceb1a8-57a0-475c-87b2-4ef5fe896621 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:32.385482 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 714/1401] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:32.405210 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-05386e5e-47a8-43f3-900e-9f29d216a86d None None] MFA Rules not processed for user `6873a3e440bd4b63b0082fc85beef49c`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:32.406499 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-05386e5e-47a8-43f3-900e-9f29d216a86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:32.469437 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:32.543368 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 688/1402] 10.222.0.83 () {58 vars in 1040 bytes} [Mon Aug 1 15:26:32 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:26:32.553264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dceb1a8-57a0-475c-87b2-4ef5fe896621 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:32.554531 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2dceb1a8-57a0-475c-87b2-4ef5fe896621 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:32.646990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dceb1a8-57a0-475c-87b2-4ef5fe896621 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:32.707419 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:32.756598 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa72ee0-fbe9-4371-997d-c7ae6bbfec0f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6873a3e440bd4b63b0082fc85beef49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:32.757274 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa72ee0-fbe9-4371-997d-c7ae6bbfec0f service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:32.757467 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa72ee0-fbe9-4371-997d-c7ae6bbfec0f service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:32.757739 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa72ee0-fbe9-4371-997d-c7ae6bbfec0f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:32.759423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0fa72ee0-fbe9-4371-997d-c7ae6bbfec0f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:32.873877 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa72ee0-fbe9-4371-997d-c7ae6bbfec0f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:32.875521 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa72ee0-fbe9-4371-997d-c7ae6bbfec0f service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:32.886574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 715/1403] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Aug 1 15:26:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:26:34.550999 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96614a-8c52-4d7c-97de-3bd741571f8f None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:34.551596 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96614a-8c52-4d7c-97de-3bd741571f8f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:34.551787 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96614a-8c52-4d7c-97de-3bd741571f8f None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:34.553272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 689/1404] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:34.558743 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8d95e-92bd-4514-93f8-30c6a760f2ff None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:34.559201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8d95e-92bd-4514-93f8-30c6a760f2ff None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:34.559305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8d95e-92bd-4514-93f8-30c6a760f2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:34.634108 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-cdd8d95e-92bd-4514-93f8-30c6a760f2ff None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:34.638785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-cdd8d95e-92bd-4514-93f8-30c6a760f2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:34.687523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:34.752273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 716/1405] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:34.798094 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96614a-8c52-4d7c-97de-3bd741571f8f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:34.799671 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3f96614a-8c52-4d7c-97de-3bd741571f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:34.972674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96614a-8c52-4d7c-97de-3bd741571f8f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:35.009467 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:35.115013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725e0fb-d43c-47fe-8d10-96bf7f29c5c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:35.115670 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725e0fb-d43c-47fe-8d10-96bf7f29c5c7 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:35.116219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725e0fb-d43c-47fe-8d10-96bf7f29c5c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:35.116530 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725e0fb-d43c-47fe-8d10-96bf7f29c5c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:35.118518 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e725e0fb-d43c-47fe-8d10-96bf7f29c5c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:35.219713 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725e0fb-d43c-47fe-8d10-96bf7f29c5c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:35.222535 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725e0fb-d43c-47fe-8d10-96bf7f29c5c7 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:35.244358 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 690/1406] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:37.330326 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4bd46-83f9-4a0c-946b-f8f8cc899f4c None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:37.330876 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4bd46-83f9-4a0c-946b-f8f8cc899f4c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:37.330876 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4bd46-83f9-4a0c-946b-f8f8cc899f4c None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:37.332677 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 717/1407] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:26:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:37.338144 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc244ed8-5698-4fa2-a989-67c92fa353b6 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:37.338604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc244ed8-5698-4fa2-a989-67c92fa353b6 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:37.338931 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc244ed8-5698-4fa2-a989-67c92fa353b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:37.383842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:37.416745 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-dc244ed8-5698-4fa2-a989-67c92fa353b6 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:37.418019 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-dc244ed8-5698-4fa2-a989-67c92fa353b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:37.541539 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 691/1408] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:26:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:37.590469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4bd46-83f9-4a0c-946b-f8f8cc899f4c None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:37.591990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b9b4bd46-83f9-4a0c-946b-f8f8cc899f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:37.765623 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4bd46-83f9-4a0c-946b-f8f8cc899f4c None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:37.794030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:37.898784 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4b2ac-7d90-4576-969c-ebdc1222f13c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:37.900154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4b2ac-7d90-4576-969c-ebdc1222f13c service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:37.900665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4b2ac-7d90-4576-969c-ebdc1222f13c service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:37.901179 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4b2ac-7d90-4576-969c-ebdc1222f13c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:37.903281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4ee4b2ac-7d90-4576-969c-ebdc1222f13c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:38.023677 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4b2ac-7d90-4576-969c-ebdc1222f13c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:38.025751 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4b2ac-7d90-4576-969c-ebdc1222f13c service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:38.047458 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 718/1409] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:47.539512 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=51 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:48.044745 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=86 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:55.325121 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179fc34-efdf-4879-ba01-fd862b9153e9 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:55.325673 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179fc34-efdf-4879-ba01-fd862b9153e9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:55.325673 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179fc34-efdf-4879-ba01-fd862b9153e9 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:55.327284 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 692/1410] 10.222.0.83 () {54 vars in 918 bytes} [Mon Aug 1 15:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:55.332457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc716a8c-c095-454b-8233-040af58cee6d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:55.332879 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc716a8c-c095-454b-8233-040af58cee6d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:55.333010 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc716a8c-c095-454b-8233-040af58cee6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:55.410133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-dc716a8c-c095-454b-8233-040af58cee6d None None] MFA Rules not processed for user `ef57a9394b114eb3ab499133b5d40781`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:55.411361 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-dc716a8c-c095-454b-8233-040af58cee6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:55.520798 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 719/1411] 10.222.0.83 () {58 vars in 1015 bytes} [Mon Aug 1 15:26:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:55.536134 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afca9a2-458a-494d-8d6b-8881cb9cbe1f None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:55.536651 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afca9a2-458a-494d-8d6b-8881cb9cbe1f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:55.537140 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afca9a2-458a-494d-8d6b-8881cb9cbe1f None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:55.539555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 693/1412] 10.222.0.83 () {54 vars in 918 bytes} [Mon Aug 1 15:26:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:55.544788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b7045-b40a-4c85-8156-4b6703e7a605 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:55.545082 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b7045-b40a-4c85-8156-4b6703e7a605 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:55.545366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b7045-b40a-4c85-8156-4b6703e7a605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:55.629332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:55.632708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-356b7045-b40a-4c85-8156-4b6703e7a605 None None] MFA Rules not processed for user `25dc41ffc41c47fdbeec06110d04610c`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:55.634061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-356b7045-b40a-4c85-8156-4b6703e7a605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:55.751696 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 720/1413] 10.222.0.83 () {58 vars in 1015 bytes} [Mon Aug 1 15:26:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:58.238934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe399d93-545d-44b6-90d8-ff27227f5267 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:58.239502 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe399d93-545d-44b6-90d8-ff27227f5267 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:58.239502 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe399d93-545d-44b6-90d8-ff27227f5267 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:58.240882 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 694/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:26:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:58.246277 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d3b7e-29d2-4650-8360-afef81a389e2 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:58.246687 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d3b7e-29d2-4650-8360-afef81a389e2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:58.246836 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d3b7e-29d2-4650-8360-afef81a389e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:58.316421 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f65d3b7e-29d2-4650-8360-afef81a389e2 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:58.318560 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f65d3b7e-29d2-4650-8360-afef81a389e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:58.395534 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:58.437875 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 721/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:58.483452 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe399d93-545d-44b6-90d8-ff27227f5267 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:58.485337 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-fe399d93-545d-44b6-90d8-ff27227f5267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:58.636036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe399d93-545d-44b6-90d8-ff27227f5267 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:58.719656 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:58.762098 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ad387-310f-4b77-b303-790af2c1514d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:58.762727 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ad387-310f-4b77-b303-790af2c1514d service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:58.763010 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ad387-310f-4b77-b303-790af2c1514d service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:58.763283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ad387-310f-4b77-b303-790af2c1514d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:58.764932 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-573ad387-310f-4b77-b303-790af2c1514d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:58.854545 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ad387-310f-4b77-b303-790af2c1514d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:58.856183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ad387-310f-4b77-b303-790af2c1514d service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:58.875162 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 695/1416] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:59.781970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886b23c-851d-4291-97d6-6e5246a3ce7c None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:59.782515 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886b23c-851d-4291-97d6-6e5246a3ce7c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:59.782515 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886b23c-851d-4291-97d6-6e5246a3ce7c None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:59.783928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 722/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:59.789397 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f940a0-ec4f-4592-8374-cbbfa50689ae None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:59.789820 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f940a0-ec4f-4592-8374-cbbfa50689ae None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:59.789966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f940a0-ec4f-4592-8374-cbbfa50689ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:59.871302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-97f940a0-ec4f-4592-8374-cbbfa50689ae None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:59.872764 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-97f940a0-ec4f-4592-8374-cbbfa50689ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:59.904964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:59.977862 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 696/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:00.132152 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886b23c-851d-4291-97d6-6e5246a3ce7c None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:00.133649 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e886b23c-851d-4291-97d6-6e5246a3ce7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:00.282356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886b23c-851d-4291-97d6-6e5246a3ce7c None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:00.322188 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:00.397341 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47aa78-ab17-4c44-9b01-b89734674556 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:00.398004 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47aa78-ab17-4c44-9b01-b89734674556 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:00.398291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47aa78-ab17-4c44-9b01-b89734674556 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:00.398566 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47aa78-ab17-4c44-9b01-b89734674556 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:00.400190 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6f47aa78-ab17-4c44-9b01-b89734674556 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:00.489517 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47aa78-ab17-4c44-9b01-b89734674556 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:00.491178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47aa78-ab17-4c44-9b01-b89734674556 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:00.510931 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 723/1419] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:01.412089 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d915009-7e90-4814-950e-5a55c4ef9a00 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:01.412653 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d915009-7e90-4814-950e-5a55c4ef9a00 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:01.412653 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d915009-7e90-4814-950e-5a55c4ef9a00 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:01.414066 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 697/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:27:01.419059 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a820b-3d3e-428b-b12a-404229f19b6c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:01.419507 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a820b-3d3e-428b-b12a-404229f19b6c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:01.419603 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a820b-3d3e-428b-b12a-404229f19b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:01.474681 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:01.497906 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1f6a820b-3d3e-428b-b12a-404229f19b6c None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:01.499607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1f6a820b-3d3e-428b-b12a-404229f19b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:01.604407 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 724/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:01.759670 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d915009-7e90-4814-950e-5a55c4ef9a00 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:01.761196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3d915009-7e90-4814-950e-5a55c4ef9a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:01.911737 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d915009-7e90-4814-950e-5a55c4ef9a00 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:01.925018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:02.028118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe452b-1b4d-4dc4-96e8-652df6ae7445 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:02.028656 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe452b-1b4d-4dc4-96e8-652df6ae7445 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:02.028938 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe452b-1b4d-4dc4-96e8-652df6ae7445 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:02.029208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe452b-1b4d-4dc4-96e8-652df6ae7445 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:02.030874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-aebe452b-1b4d-4dc4-96e8-652df6ae7445 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:02.127532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe452b-1b4d-4dc4-96e8-652df6ae7445 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:02.129174 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe452b-1b4d-4dc4-96e8-652df6ae7445 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:02.147844 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 698/1422] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:03.038273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f4f8a-7cda-446a-b119-3d31962f535a None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:03.038856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f4f8a-7cda-446a-b119-3d31962f535a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:03.038856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f4f8a-7cda-446a-b119-3d31962f535a None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:03.040234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 725/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:27:03.045310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3811887-3f7e-46ce-9a28-aaad028baf35 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:03.045725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3811887-3f7e-46ce-9a28-aaad028baf35 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:03.045872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3811887-3f7e-46ce-9a28-aaad028baf35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:03.070110 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:03.130535 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e3811887-3f7e-46ce-9a28-aaad028baf35 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:03.131785 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e3811887-3f7e-46ce-9a28-aaad028baf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:03.234011 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 699/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:03.390388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f4f8a-7cda-446a-b119-3d31962f535a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:03.391898 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-778f4f8a-7cda-446a-b119-3d31962f535a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:03.508540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:03.545707 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f4f8a-7cda-446a-b119-3d31962f535a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:03.664146 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27779a32-0bf2-430d-a84e-152858bbf7ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:03.664692 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27779a32-0bf2-430d-a84e-152858bbf7ac service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:03.664979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27779a32-0bf2-430d-a84e-152858bbf7ac service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:03.665251 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27779a32-0bf2-430d-a84e-152858bbf7ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:03.666947 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-27779a32-0bf2-430d-a84e-152858bbf7ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:03.744174 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:03.760409 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27779a32-0bf2-430d-a84e-152858bbf7ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:03.762445 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27779a32-0bf2-430d-a84e-152858bbf7ac service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:03.782228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 726/1425] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:04.671585 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e198519-b32d-4d91-ad21-a0a58a50c150 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:04.672120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e198519-b32d-4d91-ad21-a0a58a50c150 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:04.672120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e198519-b32d-4d91-ad21-a0a58a50c150 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:04.673595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 700/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:27:04.678836 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3f161-93ae-47b8-b720-1ef63c05775c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:04.679247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3f161-93ae-47b8-b720-1ef63c05775c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:04.679386 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3f161-93ae-47b8-b720-1ef63c05775c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:04.758383 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-efd3f161-93ae-47b8-b720-1ef63c05775c None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:04.759637 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-efd3f161-93ae-47b8-b720-1ef63c05775c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:04.979881 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 727/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:27:05.132282 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e198519-b32d-4d91-ad21-a0a58a50c150 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:05.133851 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8e198519-b32d-4d91-ad21-a0a58a50c150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:05.222009 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:05.287118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e198519-b32d-4d91-ad21-a0a58a50c150 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:05.399645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4ed32-675b-4f77-8959-a12d3758b7f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:05.400270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4ed32-675b-4f77-8959-a12d3758b7f4 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:05.400551 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4ed32-675b-4f77-8959-a12d3758b7f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:05.400823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4ed32-675b-4f77-8959-a12d3758b7f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:05.402485 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-dff4ed32-675b-4f77-8959-a12d3758b7f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:05.451761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:05.496430 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4ed32-675b-4f77-8959-a12d3758b7f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:05.498120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4ed32-675b-4f77-8959-a12d3758b7f4 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:05.516234 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 701/1428] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:06.334197 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8e367-365b-406f-a8fe-dccc6f92bae4 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:06.334968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8e367-365b-406f-a8fe-dccc6f92bae4 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:06.334968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8e367-365b-406f-a8fe-dccc6f92bae4 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:06.336014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 728/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:27:06.341234 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb5bfb-1e30-4bca-8d4f-c038ce1e846f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:06.341769 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb5bfb-1e30-4bca-8d4f-c038ce1e846f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:06.341920 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb5bfb-1e30-4bca-8d4f-c038ce1e846f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:06.418523 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-48bb5bfb-1e30-4bca-8d4f-c038ce1e846f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:06.419921 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-48bb5bfb-1e30-4bca-8d4f-c038ce1e846f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:06.519163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 702/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:06.672198 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8e367-365b-406f-a8fe-dccc6f92bae4 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:06.673733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b3e8e367-365b-406f-a8fe-dccc6f92bae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:06.721710 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:06.824667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8e367-365b-406f-a8fe-dccc6f92bae4 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:06.941039 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5c77d-dcee-4b04-9fb4-f9080a592f0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:06.941569 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5c77d-dcee-4b04-9fb4-f9080a592f0b service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:06.941845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5c77d-dcee-4b04-9fb4-f9080a592f0b service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:06.942147 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5c77d-dcee-4b04-9fb4-f9080a592f0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:06.943789 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-bfb5c77d-dcee-4b04-9fb4-f9080a592f0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:06.950865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:07.034493 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5c77d-dcee-4b04-9fb4-f9080a592f0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:07.036075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5c77d-dcee-4b04-9fb4-f9080a592f0b service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:07.055282 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 729/1431] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:07.897616 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354da0e9-38de-460c-bbcc-9edb348b294b None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:07.898448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354da0e9-38de-460c-bbcc-9edb348b294b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:07.898448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354da0e9-38de-460c-bbcc-9edb348b294b None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:07.899634 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 703/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:27:07.904786 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a004d3-1457-471c-aaaa-25f85b4136ec None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:07.905219 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a004d3-1457-471c-aaaa-25f85b4136ec None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:07.905322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a004d3-1457-471c-aaaa-25f85b4136ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:07.981066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c8a004d3-1457-471c-aaaa-25f85b4136ec None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:07.982319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c8a004d3-1457-471c-aaaa-25f85b4136ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:08.050113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:08.081558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 730/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:08.233219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354da0e9-38de-460c-bbcc-9edb348b294b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:08.234709 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-354da0e9-38de-460c-bbcc-9edb348b294b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:08.253635 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:08.386057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354da0e9-38de-460c-bbcc-9edb348b294b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:08.474403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:08.501105 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc21056-0550-4bb6-94ba-e261e2d4b92d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:08.501656 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc21056-0550-4bb6-94ba-e261e2d4b92d service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:08.501961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc21056-0550-4bb6-94ba-e261e2d4b92d service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:08.502266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc21056-0550-4bb6-94ba-e261e2d4b92d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:08.504466 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7dc21056-0550-4bb6-94ba-e261e2d4b92d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:08.595823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc21056-0550-4bb6-94ba-e261e2d4b92d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:08.598151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc21056-0550-4bb6-94ba-e261e2d4b92d service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:08.616636 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 704/1434] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:09.426384 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d8fc7-d471-40ad-a5a3-13bdcf9f4111 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:09.426946 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d8fc7-d471-40ad-a5a3-13bdcf9f4111 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:09.426946 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d8fc7-d471-40ad-a5a3-13bdcf9f4111 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:09.428300 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 731/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:27:09.433490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91819a31-d657-44ab-878f-2b16370445a5 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:09.433937 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91819a31-d657-44ab-878f-2b16370445a5 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:09.434081 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91819a31-d657-44ab-878f-2b16370445a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:09.512720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-91819a31-d657-44ab-878f-2b16370445a5 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:09.513968 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-91819a31-d657-44ab-878f-2b16370445a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:09.556991 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:09.613911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 705/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:09.763172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d8fc7-d471-40ad-a5a3-13bdcf9f4111 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:09.764626 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ca0d8fc7-d471-40ad-a5a3-13bdcf9f4111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:09.913309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d8fc7-d471-40ad-a5a3-13bdcf9f4111 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:09.972359 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:10.035065 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44166b6f-e6e4-4e1f-ac04-1f733939695c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:10.035591 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44166b6f-e6e4-4e1f-ac04-1f733939695c service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:10.035921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44166b6f-e6e4-4e1f-ac04-1f733939695c service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:10.036132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44166b6f-e6e4-4e1f-ac04-1f733939695c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:10.037766 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-44166b6f-e6e4-4e1f-ac04-1f733939695c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:10.128813 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44166b6f-e6e4-4e1f-ac04-1f733939695c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:10.130448 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44166b6f-e6e4-4e1f-ac04-1f733939695c service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:10.148969 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 732/1437] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:10.954629 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62bd62-c206-4e0f-b113-4f8d17cf3efe None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:10.955187 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62bd62-c206-4e0f-b113-4f8d17cf3efe None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:10.955187 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62bd62-c206-4e0f-b113-4f8d17cf3efe None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:10.956584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 706/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:27:10.962017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8146fb-1ee1-4422-b66e-ad9aaf6d06d0 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:10.962439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8146fb-1ee1-4422-b66e-ad9aaf6d06d0 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:10.962552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8146fb-1ee1-4422-b66e-ad9aaf6d06d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:11.040579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-fc8146fb-1ee1-4422-b66e-ad9aaf6d06d0 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:11.041787 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fc8146fb-1ee1-4422-b66e-ad9aaf6d06d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:11.043834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:11.144759 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 733/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:27:11.298577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62bd62-c206-4e0f-b113-4f8d17cf3efe None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:11.300086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0f62bd62-c206-4e0f-b113-4f8d17cf3efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:11.444980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62bd62-c206-4e0f-b113-4f8d17cf3efe None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:11.472839 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:11.577445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841208fa-b205-4a24-9ab2-6a213f949fc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:11.578102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841208fa-b205-4a24-9ab2-6a213f949fc7 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:11.578389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841208fa-b205-4a24-9ab2-6a213f949fc7 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:11.578666 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841208fa-b205-4a24-9ab2-6a213f949fc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:11.580304 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-841208fa-b205-4a24-9ab2-6a213f949fc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:11.669838 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841208fa-b205-4a24-9ab2-6a213f949fc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:11.671488 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841208fa-b205-4a24-9ab2-6a213f949fc7 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:11.691194 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 707/1440] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:13.632443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296fc53-9914-4b4c-8cc7-fbad5570d06a None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:13.633325 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296fc53-9914-4b4c-8cc7-fbad5570d06a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:13.633325 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296fc53-9914-4b4c-8cc7-fbad5570d06a None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:13.634459 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 734/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:27:13.640352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359168e8-33ef-40b0-94e4-af458b5cf041 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:13.640794 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359168e8-33ef-40b0-94e4-af458b5cf041 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:13.640898 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359168e8-33ef-40b0-94e4-af458b5cf041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:13.682857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:13.708415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-359168e8-33ef-40b0-94e4-af458b5cf041 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:13.709721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-359168e8-33ef-40b0-94e4-af458b5cf041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:13.806718 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 708/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:27:13.845113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296fc53-9914-4b4c-8cc7-fbad5570d06a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:13.847375 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e296fc53-9914-4b4c-8cc7-fbad5570d06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:13.964133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296fc53-9914-4b4c-8cc7-fbad5570d06a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:13.996424 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:14.038966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2d240-c1de-4a36-96f6-1031fcfc8678 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:14.039589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2d240-c1de-4a36-96f6-1031fcfc8678 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:14.039873 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2d240-c1de-4a36-96f6-1031fcfc8678 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:14.040139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2d240-c1de-4a36-96f6-1031fcfc8678 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:14.041791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-11d2d240-c1de-4a36-96f6-1031fcfc8678 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:14.150610 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2d240-c1de-4a36-96f6-1031fcfc8678 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:14.152036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2d240-c1de-4a36-96f6-1031fcfc8678 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:14.170848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 735/1443] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:16.341378 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bd71e-b39f-4723-a2fe-a394c18572f4 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:16.341943 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bd71e-b39f-4723-a2fe-a394c18572f4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:16.341943 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bd71e-b39f-4723-a2fe-a394c18572f4 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:16.343291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 709/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:27:16.348689 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d658de2-5f6e-491d-8743-e2b6cc9a2a71 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:16.349085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d658de2-5f6e-491d-8743-e2b6cc9a2a71 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:16.349237 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d658de2-5f6e-491d-8743-e2b6cc9a2a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:16.416386 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-5d658de2-5f6e-491d-8743-e2b6cc9a2a71 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:16.417608 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5d658de2-5f6e-491d-8743-e2b6cc9a2a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:16.431001 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:16.518992 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 736/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:27:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:16.559287 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0bd71e-b39f-4723-a2fe-a394c18572f4 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:16.560821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bd0bd71e-b39f-4723-a2fe-a394c18572f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:16.670114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0bd71e-b39f-4723-a2fe-a394c18572f4 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:16.682351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:16.744809 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7218b5-a54f-440d-a632-93ed3f2db039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:16.745434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7218b5-a54f-440d-a632-93ed3f2db039 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:16.745742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7218b5-a54f-440d-a632-93ed3f2db039 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:16.746182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7218b5-a54f-440d-a632-93ed3f2db039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:16.747852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7a7218b5-a54f-440d-a632-93ed3f2db039 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:16.854890 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7218b5-a54f-440d-a632-93ed3f2db039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:16.856329 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7218b5-a54f-440d-a632-93ed3f2db039 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:16.876191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 710/1446] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:19.304962 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c0eb8-9730-49ef-bfbc-cc29ac30f2fd None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:19.305581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c0eb8-9730-49ef-bfbc-cc29ac30f2fd None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:19.305581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c0eb8-9730-49ef-bfbc-cc29ac30f2fd None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:19.306901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 737/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:27:19.312468 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ed8cf-e59e-4f2a-802a-b678c0fa34ef None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:19.312885 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ed8cf-e59e-4f2a-802a-b678c0fa34ef None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:19.313027 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ed8cf-e59e-4f2a-802a-b678c0fa34ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:19.361331 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:19.379378 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-b70ed8cf-e59e-4f2a-802a-b678c0fa34ef None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:19.380725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b70ed8cf-e59e-4f2a-802a-b678c0fa34ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:19.480628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 711/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:27:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:19.522271 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c0eb8-9730-49ef-bfbc-cc29ac30f2fd None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:19.523802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-603c0eb8-9730-49ef-bfbc-cc29ac30f2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:19.628499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c0eb8-9730-49ef-bfbc-cc29ac30f2fd None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:19.669745 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:19.706607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60e45a-ddd8-44b8-b3f8-c88cbf6a1c82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:19.707226 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60e45a-ddd8-44b8-b3f8-c88cbf6a1c82 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:19.707502 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60e45a-ddd8-44b8-b3f8-c88cbf6a1c82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:19.707771 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60e45a-ddd8-44b8-b3f8-c88cbf6a1c82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:19.709389 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5c60e45a-ddd8-44b8-b3f8-c88cbf6a1c82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:19.821369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60e45a-ddd8-44b8-b3f8-c88cbf6a1c82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:19.822834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60e45a-ddd8-44b8-b3f8-c88cbf6a1c82 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:19.842111 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 738/1449] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:22.016624 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889f169-285b-4b40-bc15-4263b750b0f4 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:22.017270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889f169-285b-4b40-bc15-4263b750b0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:22.017270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889f169-285b-4b40-bc15-4263b750b0f4 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:22.019389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 712/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:27:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:27:22.025023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00aa4c-1e49-4343-b566-86e8d7904013 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:22.025478 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00aa4c-1e49-4343-b566-86e8d7904013 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:22.025598 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00aa4c-1e49-4343-b566-86e8d7904013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:22.092362 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1a00aa4c-1e49-4343-b566-86e8d7904013 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:22.093569 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1a00aa4c-1e49-4343-b566-86e8d7904013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:22.113579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:22.194761 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 739/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:27:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:22.239670 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889f169-285b-4b40-bc15-4263b750b0f4 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:22.241225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2889f169-285b-4b40-bc15-4263b750b0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:22.351119 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889f169-285b-4b40-bc15-4263b750b0f4 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:22.372271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:22.427346 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd4cec-0d43-4e7b-9129-ff6be82a99b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:22.427978 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd4cec-0d43-4e7b-9129-ff6be82a99b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:22.428265 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd4cec-0d43-4e7b-9129-ff6be82a99b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:22.428542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd4cec-0d43-4e7b-9129-ff6be82a99b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:22.430241 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ecbd4cec-0d43-4e7b-9129-ff6be82a99b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:22.544628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd4cec-0d43-4e7b-9129-ff6be82a99b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:22.546843 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd4cec-0d43-4e7b-9129-ff6be82a99b0 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:22.569431 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 713/1452] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:24.721648 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b40ef-3a04-4fd3-a479-472cf763c468 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:24.722205 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b40ef-3a04-4fd3-a479-472cf763c468 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:24.722205 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b40ef-3a04-4fd3-a479-472cf763c468 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:24.723516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 740/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:27:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:27:24.729064 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfde19-6be7-4cc5-a2d4-23e622e301a4 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:24.729496 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfde19-6be7-4cc5-a2d4-23e622e301a4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:24.729628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfde19-6be7-4cc5-a2d4-23e622e301a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:24.796495 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:24.799467 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-80bfde19-6be7-4cc5-a2d4-23e622e301a4 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:24.801299 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-80bfde19-6be7-4cc5-a2d4-23e622e301a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:24.898829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 714/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:24.946581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b40ef-3a04-4fd3-a479-472cf763c468 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:24.948213 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c99b40ef-3a04-4fd3-a479-472cf763c468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:25.052030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b40ef-3a04-4fd3-a479-472cf763c468 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:25.101877 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:25.126035 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3babdf-644a-4cdd-92ed-c922172eb75a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:25.126684 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3babdf-644a-4cdd-92ed-c922172eb75a service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:25.127450 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3babdf-644a-4cdd-92ed-c922172eb75a service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:25.127721 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3babdf-644a-4cdd-92ed-c922172eb75a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:25.129341 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ed3babdf-644a-4cdd-92ed-c922172eb75a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:25.242909 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3babdf-644a-4cdd-92ed-c922172eb75a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:25.244346 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3babdf-644a-4cdd-92ed-c922172eb75a service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:25.262709 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 741/1455] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:27.402895 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc6bf4-5753-41ee-9ee2-ae1b5341ecb1 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:27.403443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc6bf4-5753-41ee-9ee2-ae1b5341ecb1 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:27.403443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc6bf4-5753-41ee-9ee2-ae1b5341ecb1 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:27.404810 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 715/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:27:27.410716 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e30cb-5dec-4642-bd67-12b054b4b4c2 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:27.411525 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e30cb-5dec-4642-bd67-12b054b4b4c2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:27.411525 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e30cb-5dec-4642-bd67-12b054b4b4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:27.482538 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e36e30cb-5dec-4642-bd67-12b054b4b4c2 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:27.483770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e36e30cb-5dec-4642-bd67-12b054b4b4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:27.514062 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:27.587135 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 742/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:27.627268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc6bf4-5753-41ee-9ee2-ae1b5341ecb1 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:27.628813 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a1fc6bf4-5753-41ee-9ee2-ae1b5341ecb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:27.734649 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc6bf4-5753-41ee-9ee2-ae1b5341ecb1 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:27.763148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:27.810406 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665bfb5-124c-488c-a77e-4257c8c49c77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:27.811036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665bfb5-124c-488c-a77e-4257c8c49c77 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:27.811359 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665bfb5-124c-488c-a77e-4257c8c49c77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:27.811634 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665bfb5-124c-488c-a77e-4257c8c49c77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:27.813326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1665bfb5-124c-488c-a77e-4257c8c49c77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:27.929844 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665bfb5-124c-488c-a77e-4257c8c49c77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:27.931329 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665bfb5-124c-488c-a77e-4257c8c49c77 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:27.949302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 716/1458] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:30.042629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7c2be-afee-4177-a21b-966c79e1e66b None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:30.043211 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7c2be-afee-4177-a21b-966c79e1e66b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:30.043211 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7c2be-afee-4177-a21b-966c79e1e66b None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:30.044540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 743/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:27:30.050288 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc18da-697e-4b7b-804e-b68f0705a8d6 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:30.050749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc18da-697e-4b7b-804e-b68f0705a8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:30.050867 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc18da-697e-4b7b-804e-b68f0705a8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:30.119496 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-66cc18da-697e-4b7b-804e-b68f0705a8d6 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:30.120750 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-66cc18da-697e-4b7b-804e-b68f0705a8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:30.129170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:30.224489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 717/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:27:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:27:30.261641 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7c2be-afee-4177-a21b-966c79e1e66b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:30.263245 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c7a7c2be-afee-4177-a21b-966c79e1e66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:30.367210 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7c2be-afee-4177-a21b-966c79e1e66b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:30.422696 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:30.441608 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b7d88-bec9-4849-a28d-d7a13de1fda9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:30.442263 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b7d88-bec9-4849-a28d-d7a13de1fda9 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:30.442548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b7d88-bec9-4849-a28d-d7a13de1fda9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:30.442818 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b7d88-bec9-4849-a28d-d7a13de1fda9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:30.444444 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2f7b7d88-bec9-4849-a28d-d7a13de1fda9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:30.555547 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b7d88-bec9-4849-a28d-d7a13de1fda9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:30.556974 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b7d88-bec9-4849-a28d-d7a13de1fda9 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:30.575340 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 744/1461] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:32.689023 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d9f54-def4-404c-a9e3-ba71d393ff3e None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:32.689626 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d9f54-def4-404c-a9e3-ba71d393ff3e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:32.689626 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d9f54-def4-404c-a9e3-ba71d393ff3e None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:32.690999 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 718/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:27:32.699284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe2cc5-ad3a-49c2-a92a-25c67c6564ac None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:32.699822 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe2cc5-ad3a-49c2-a92a-25c67c6564ac None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:32.699822 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe2cc5-ad3a-49c2-a92a-25c67c6564ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:32.766192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e5fe2cc5-ad3a-49c2-a92a-25c67c6564ac None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:32.767425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e5fe2cc5-ad3a-49c2-a92a-25c67c6564ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:32.805364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:32.864312 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 745/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:32.904317 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d9f54-def4-404c-a9e3-ba71d393ff3e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:32.905933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3b9d9f54-def4-404c-a9e3-ba71d393ff3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:33.008921 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d9f54-def4-404c-a9e3-ba71d393ff3e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:33.046915 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:33.084262 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066d192-3c40-4d79-a702-38496fbd32cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:33.084879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066d192-3c40-4d79-a702-38496fbd32cd service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:33.085157 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066d192-3c40-4d79-a702-38496fbd32cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:33.085423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066d192-3c40-4d79-a702-38496fbd32cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:33.087065 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c066d192-3c40-4d79-a702-38496fbd32cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:33.196072 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066d192-3c40-4d79-a702-38496fbd32cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:33.197526 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066d192-3c40-4d79-a702-38496fbd32cd service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:33.216484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 719/1464] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:35.334650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d3a60-29c3-4eb0-a42c-795458f06471 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:35.335201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d3a60-29c3-4eb0-a42c-795458f06471 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:35.335201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d3a60-29c3-4eb0-a42c-795458f06471 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:35.336482 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 746/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:27:35.341818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9637c5-768d-4b44-91bc-36591356ec02 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:35.342378 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9637c5-768d-4b44-91bc-36591356ec02 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:35.342474 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9637c5-768d-4b44-91bc-36591356ec02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:35.417156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-0b9637c5-768d-4b44-91bc-36591356ec02 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:35.418410 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0b9637c5-768d-4b44-91bc-36591356ec02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:35.436635 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:35.643165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 720/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:27:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:35.681109 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d3a60-29c3-4eb0-a42c-795458f06471 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:35.682686 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-486d3a60-29c3-4eb0-a42c-795458f06471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:35.790743 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d3a60-29c3-4eb0-a42c-795458f06471 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:35.857478 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:35.864664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035524d-1a33-45ba-b454-a45f564585f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:35.865283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035524d-1a33-45ba-b454-a45f564585f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:35.865563 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035524d-1a33-45ba-b454-a45f564585f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:35.865834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035524d-1a33-45ba-b454-a45f564585f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:35.867496 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f035524d-1a33-45ba-b454-a45f564585f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:35.987470 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f035524d-1a33-45ba-b454-a45f564585f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:35.988825 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f035524d-1a33-45ba-b454-a45f564585f1 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:36.007590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 747/1467] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:38.126206 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8499dcc-9982-4f79-a25e-2931cef9536b None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:38.127355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8499dcc-9982-4f79-a25e-2931cef9536b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:38.127458 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8499dcc-9982-4f79-a25e-2931cef9536b None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:38.128837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 721/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:27:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:27:38.134300 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f440c0-4e2a-4566-8b40-be07ae774b9a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:38.134706 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f440c0-4e2a-4566-8b40-be07ae774b9a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:38.134857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f440c0-4e2a-4566-8b40-be07ae774b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:38.201042 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d6f440c0-4e2a-4566-8b40-be07ae774b9a None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:38.202568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d6f440c0-4e2a-4566-8b40-be07ae774b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:38.249795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:38.299280 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 748/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:38.339151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8499dcc-9982-4f79-a25e-2931cef9536b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:38.340715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f8499dcc-9982-4f79-a25e-2931cef9536b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:38.444533 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8499dcc-9982-4f79-a25e-2931cef9536b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:38.493986 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:38.525677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f87c5a-72a8-40c1-a554-6b49fee4fb87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:38.526675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f87c5a-72a8-40c1-a554-6b49fee4fb87 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:38.526675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f87c5a-72a8-40c1-a554-6b49fee4fb87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:38.527059 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f87c5a-72a8-40c1-a554-6b49fee4fb87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:38.528502 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-73f87c5a-72a8-40c1-a554-6b49fee4fb87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:38.639748 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f87c5a-72a8-40c1-a554-6b49fee4fb87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:38.641197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f87c5a-72a8-40c1-a554-6b49fee4fb87 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:38.660323 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 722/1470] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:48.296887 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=22 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:48.657724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=66 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:58.448768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45128fce-77fe-4113-9efc-eb1fa1437cd5 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:58.449339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45128fce-77fe-4113-9efc-eb1fa1437cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:58.449339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45128fce-77fe-4113-9efc-eb1fa1437cd5 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:58.450719 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 749/1471] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:27:58.456190 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa77ed-1863-4ceb-9963-6c0cd9c38974 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:58.456610 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa77ed-1863-4ceb-9963-6c0cd9c38974 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:58.456753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa77ed-1863-4ceb-9963-6c0cd9c38974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:58.522133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-befa77ed-1863-4ceb-9963-6c0cd9c38974 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:58.523352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-befa77ed-1863-4ceb-9963-6c0cd9c38974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:58.620071 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 723/1472] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:27:58.704323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24225fe3-bc18-4252-8b40-67e1abe1a5ca None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:58.704612 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24225fe3-bc18-4252-8b40-67e1abe1a5ca None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:58.704891 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24225fe3-bc18-4252-8b40-67e1abe1a5ca None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:58.706282 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 750/1473] 10.222.0.83 () {54 vars in 966 bytes} [Mon Aug 1 15:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:27:58.711299 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b9183-4c4b-4231-8ae3-305472c78389 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:58.711590 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b9183-4c4b-4231-8ae3-305472c78389 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:58.711899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b9183-4c4b-4231-8ae3-305472c78389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:58.787675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-b91b9183-4c4b-4231-8ae3-305472c78389 None None] MFA Rules not processed for user `69d216a568b546589e854c2b4476efe6`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:58.788894 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b91b9183-4c4b-4231-8ae3-305472c78389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:58.809749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:58.865506 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 724/1474] 10.222.0.83 () {58 vars in 1063 bytes} [Mon Aug 1 15:27:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:58.873171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24225fe3-bc18-4252-8b40-67e1abe1a5ca None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:58.874778 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-24225fe3-bc18-4252-8b40-67e1abe1a5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:58.981223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24225fe3-bc18-4252-8b40-67e1abe1a5ca None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:59.054959 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad350216-d4e0-484b-965f-103820526b47 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69d216a568b546589e854c2b4476efe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:59.055409 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad350216-d4e0-484b-965f-103820526b47 service glance] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:59.055691 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad350216-d4e0-484b-965f-103820526b47 service glance] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:59.055961 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad350216-d4e0-484b-965f-103820526b47 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:59.057576 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ad350216-d4e0-484b-965f-103820526b47 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:59.113788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:59.168409 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad350216-d4e0-484b-965f-103820526b47 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:59.169840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad350216-d4e0-484b-965f-103820526b47 service glance] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:59.189462 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 751/1475] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:01.303012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91812779-6fda-47f0-9217-6d6c5d98c558 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:01.303570 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91812779-6fda-47f0-9217-6d6c5d98c558 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:01.303729 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91812779-6fda-47f0-9217-6d6c5d98c558 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:01.304964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 725/1476] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:28:01.310406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310829ff-a0ec-4da0-887f-008a4d5abd14 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:01.310815 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310829ff-a0ec-4da0-887f-008a4d5abd14 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:01.310958 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310829ff-a0ec-4da0-887f-008a4d5abd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:01.375993 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-310829ff-a0ec-4da0-887f-008a4d5abd14 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:01.377824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-310829ff-a0ec-4da0-887f-008a4d5abd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:01.472038 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 752/1477] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:28:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:01.495825 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91812779-6fda-47f0-9217-6d6c5d98c558 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:01.497382 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-91812779-6fda-47f0-9217-6d6c5d98c558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:01.627086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91812779-6fda-47f0-9217-6d6c5d98c558 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:01.675700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:01.722458 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503946e-c0cc-4c86-8fa1-800bf7c508f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:01.723079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503946e-c0cc-4c86-8fa1-800bf7c508f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:01.723357 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503946e-c0cc-4c86-8fa1-800bf7c508f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:01.723626 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503946e-c0cc-4c86-8fa1-800bf7c508f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:01.724311 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503946e-c0cc-4c86-8fa1-800bf7c508f4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:01.726111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: WARNING py.warnings [None req-0503946e-c0cc-4c86-8fa1-800bf7c508f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:28:01.726111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: warnings.warn(msg) Aug 01 15:28:01.726111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: Aug 01 15:28:01.727233 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503946e-c0cc-4c86-8fa1-800bf7c508f4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:01.746078 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 726/1478] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Aug 1 15:28:01 2022] GET /identity/v3/projects => generated 2553 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:03.807772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5786102-180d-4fd1-b8c1-f8bcfc20dba3 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:03.808360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5786102-180d-4fd1-b8c1-f8bcfc20dba3 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:03.808454 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5786102-180d-4fd1-b8c1-f8bcfc20dba3 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:03.809786 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 753/1479] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:28:03.815803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb97b4-ce9d-4b2a-9053-21d29a42d1ad None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:03.816394 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb97b4-ce9d-4b2a-9053-21d29a42d1ad None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:03.816394 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb97b4-ce9d-4b2a-9053-21d29a42d1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:03.883561 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-69cb97b4-ce9d-4b2a-9053-21d29a42d1ad None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:03.884838 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-69cb97b4-ce9d-4b2a-9053-21d29a42d1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:03.988390 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 727/1480] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:04.030440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5786102-180d-4fd1-b8c1-f8bcfc20dba3 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:04.032522 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d5786102-180d-4fd1-b8c1-f8bcfc20dba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:04.042347 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:04.181730 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5786102-180d-4fd1-b8c1-f8bcfc20dba3 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:04.274909 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:04.308707 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6944ab-59fc-44b5-a869-82bf7d7fb0fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:04.309347 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6944ab-59fc-44b5-a869-82bf7d7fb0fd service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:04.309624 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6944ab-59fc-44b5-a869-82bf7d7fb0fd service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:04.309919 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6944ab-59fc-44b5-a869-82bf7d7fb0fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:04.311533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8b6944ab-59fc-44b5-a869-82bf7d7fb0fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:04.399222 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6944ab-59fc-44b5-a869-82bf7d7fb0fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:04.400820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6944ab-59fc-44b5-a869-82bf7d7fb0fd service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:04.419214 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 754/1481] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:28:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:06.654818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3b408-c2fc-422f-a4ec-fb3506bfa94d None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:06.655687 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3b408-c2fc-422f-a4ec-fb3506bfa94d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:06.655687 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3b408-c2fc-422f-a4ec-fb3506bfa94d None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:06.656833 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 728/1482] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:28:06.663624 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877c94a-2779-4a42-8c77-6918f7dd3d90 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:06.664089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877c94a-2779-4a42-8c77-6918f7dd3d90 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:06.664197 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877c94a-2779-4a42-8c77-6918f7dd3d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:06.733595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-5877c94a-2779-4a42-8c77-6918f7dd3d90 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:06.734758 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5877c94a-2779-4a42-8c77-6918f7dd3d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:06.781188 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:06.832359 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 755/1483] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:06.868534 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3b408-c2fc-422f-a4ec-fb3506bfa94d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:06.870126 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-14e3b408-c2fc-422f-a4ec-fb3506bfa94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:06.902110 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:06.983996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3b408-c2fc-422f-a4ec-fb3506bfa94d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:07.065111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa33ccd-feef-4962-b081-c47abbb78357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:07.065803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa33ccd-feef-4962-b081-c47abbb78357 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:07.066122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa33ccd-feef-4962-b081-c47abbb78357 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:07.066402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa33ccd-feef-4962-b081-c47abbb78357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:07.068051 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0fa33ccd-feef-4962-b081-c47abbb78357 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:07.143433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:07.179425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa33ccd-feef-4962-b081-c47abbb78357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:07.180874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa33ccd-feef-4962-b081-c47abbb78357 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:07.198949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 729/1484] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:28:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:09.295207 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35db19-9df5-4f71-b560-720287ab9f78 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:09.295826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35db19-9df5-4f71-b560-720287ab9f78 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:09.295826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35db19-9df5-4f71-b560-720287ab9f78 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:09.297132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 756/1485] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:28:09.302984 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f6e5d-8a3b-4d78-9596-44fc29643c7b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:09.303446 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f6e5d-8a3b-4d78-9596-44fc29643c7b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:09.303850 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f6e5d-8a3b-4d78-9596-44fc29643c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:09.369121 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-984f6e5d-8a3b-4d78-9596-44fc29643c7b None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:09.370348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-984f6e5d-8a3b-4d78-9596-44fc29643c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:09.468050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 730/1486] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:09.506397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f35db19-9df5-4f71-b560-720287ab9f78 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:09.507972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5f35db19-9df5-4f71-b560-720287ab9f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:09.612748 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f35db19-9df5-4f71-b560-720287ab9f78 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:09.688817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:09.691209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc23f6-2b4a-498e-b563-f23a58188934 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:09.692031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc23f6-2b4a-498e-b563-f23a58188934 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:09.692485 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc23f6-2b4a-498e-b563-f23a58188934 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:09.692975 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc23f6-2b4a-498e-b563-f23a58188934 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:09.694809 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-afcc23f6-2b4a-498e-b563-f23a58188934 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:09.816163 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc23f6-2b4a-498e-b563-f23a58188934 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:09.817613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc23f6-2b4a-498e-b563-f23a58188934 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:09.835615 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 757/1487] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:28:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:11.992772 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f61cf-5dd2-463a-be3e-fe41c82abfe8 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:11.993349 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f61cf-5dd2-463a-be3e-fe41c82abfe8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:11.993472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f61cf-5dd2-463a-be3e-fe41c82abfe8 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:11.994961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 731/1488] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:28:12.001438 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d2629-7573-46f1-998f-bdb7b099ca27 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:12.002261 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d2629-7573-46f1-998f-bdb7b099ca27 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:12.002261 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d2629-7573-46f1-998f-bdb7b099ca27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:12.070963 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-611d2629-7573-46f1-998f-bdb7b099ca27 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:12.072185 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-611d2629-7573-46f1-998f-bdb7b099ca27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:12.125549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:12.174340 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 758/1489] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:12.214882 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f61cf-5dd2-463a-be3e-fe41c82abfe8 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:12.216407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bd5f61cf-5dd2-463a-be3e-fe41c82abfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:12.253000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:12.320846 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f61cf-5dd2-463a-be3e-fe41c82abfe8 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:12.396743 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902e578-d737-4e31-b5f9-b1ac3d44f68a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:12.397364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902e578-d737-4e31-b5f9-b1ac3d44f68a service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:12.397720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902e578-d737-4e31-b5f9-b1ac3d44f68a service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:12.397942 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902e578-d737-4e31-b5f9-b1ac3d44f68a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:12.399563 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9902e578-d737-4e31-b5f9-b1ac3d44f68a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:12.486216 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:12.517168 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902e578-d737-4e31-b5f9-b1ac3d44f68a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:12.518552 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902e578-d737-4e31-b5f9-b1ac3d44f68a service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:12.537462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 732/1490] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:28:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:14.783673 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db95752-5cae-4332-bf19-40a404b185f9 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:14.784209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db95752-5cae-4332-bf19-40a404b185f9 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:14.784209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db95752-5cae-4332-bf19-40a404b185f9 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:14.785560 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 759/1491] 10.222.0.83 () {54 vars in 926 bytes} [Mon Aug 1 15:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:28:14.790856 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52143361-005f-4246-b592-68693153f3ca None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:14.791260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52143361-005f-4246-b592-68693153f3ca None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:14.791420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52143361-005f-4246-b592-68693153f3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:14.857116 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-52143361-005f-4246-b592-68693153f3ca None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:14.858345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-52143361-005f-4246-b592-68693153f3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:14.957848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 733/1492] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Aug 1 15:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:14.997072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db95752-5cae-4332-bf19-40a404b185f9 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:14.998659 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4db95752-5cae-4332-bf19-40a404b185f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:15.112989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db95752-5cae-4332-bf19-40a404b185f9 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:15.187149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b1a2a-cc0d-4ae9-99f9-22a3e1e0e864 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:15.187774 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b1a2a-cc0d-4ae9-99f9-22a3e1e0e864 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:15.188054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b1a2a-cc0d-4ae9-99f9-22a3e1e0e864 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:15.188327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b1a2a-cc0d-4ae9-99f9-22a3e1e0e864 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:15.189997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-195b1a2a-cc0d-4ae9-99f9-22a3e1e0e864 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:15.199072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:15.299101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b1a2a-cc0d-4ae9-99f9-22a3e1e0e864 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:15.300531 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b1a2a-cc0d-4ae9-99f9-22a3e1e0e864 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:15.318271 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 760/1493] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:28:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:24.956027 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=82 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:25.315966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=50 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:45.411032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71371079-00de-4f8e-ae0d-f11f0f671f55 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:45.411633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71371079-00de-4f8e-ae0d-f11f0f671f55 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:45.411633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71371079-00de-4f8e-ae0d-f11f0f671f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:45.489679 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-71371079-00de-4f8e-ae0d-f11f0f671f55 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:45.490963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-71371079-00de-4f8e-ae0d-f11f0f671f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:45.598262 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 734/1494] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:45.606775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761abc14-4cfb-4323-9d3e-f7ae9d8bffcc None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:45.607360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761abc14-4cfb-4323-9d3e-f7ae9d8bffcc None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:45.607360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761abc14-4cfb-4323-9d3e-f7ae9d8bffcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:45.690459 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-761abc14-4cfb-4323-9d3e-f7ae9d8bffcc None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:45.691596 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-761abc14-4cfb-4323-9d3e-f7ae9d8bffcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:45.789296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 761/1495] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:45.794150 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71371079-00de-4f8e-ae0d-f11f0f671f55 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:45.795478 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-71371079-00de-4f8e-ae0d-f11f0f671f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:45.807703 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71371079-00de-4f8e-ae0d-f11f0f671f55 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:45.868677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:45.904253 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a056a6b8-d225-4300-b755-b7182041e397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:45.905495 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056a6b8-d225-4300-b755-b7182041e397 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:45.905775 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056a6b8-d225-4300-b755-b7182041e397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:45.906086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056a6b8-d225-4300-b755-b7182041e397 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:45.906785 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a056a6b8-d225-4300-b755-b7182041e397 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:29:45.907083 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a056a6b8-d225-4300-b755-b7182041e397 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:45.908115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: WARNING py.warnings [None req-a056a6b8-d225-4300-b755-b7182041e397 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:29:45.908115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: warnings.warn(msg) Aug 01 15:29:45.908115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: Aug 01 15:29:45.908994 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a056a6b8-d225-4300-b755-b7182041e397 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:45.920169 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 735/1496] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:29:45.928531 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761abc14-4cfb-4323-9d3e-f7ae9d8bffcc None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:45.938625 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761abc14-4cfb-4323-9d3e-f7ae9d8bffcc None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:46.000403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:46.035050 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fea81-e63d-42ba-bb46-2226b64f8100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:46.035665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fea81-e63d-42ba-bb46-2226b64f8100 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:46.035943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fea81-e63d-42ba-bb46-2226b64f8100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:46.036210 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fea81-e63d-42ba-bb46-2226b64f8100 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:46.036870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fea81-e63d-42ba-bb46-2226b64f8100 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:46.038631 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: WARNING py.warnings [None req-401fea81-e63d-42ba-bb46-2226b64f8100 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:29:46.038631 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: warnings.warn(msg) Aug 01 15:29:46.038631 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: Aug 01 15:29:46.039804 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fea81-e63d-42ba-bb46-2226b64f8100 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:46.099140 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 762/1497] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:29:45 2022] POST /identity/v3/projects => generated 377 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:29:46.103375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a056a6b8-d225-4300-b755-b7182041e397 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:46.104981 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a056a6b8-d225-4300-b755-b7182041e397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:46.139878 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a056a6b8-d225-4300-b755-b7182041e397 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:46.233490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd6a43-d1c4-48e3-a355-a99aa9fa4bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:46.234138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd6a43-d1c4-48e3-a355-a99aa9fa4bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:46.234416 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd6a43-d1c4-48e3-a355-a99aa9fa4bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:46.234685 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd6a43-d1c4-48e3-a355-a99aa9fa4bb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:46.235371 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd6a43-d1c4-48e3-a355-a99aa9fa4bb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:46.236423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: WARNING py.warnings [None req-96fd6a43-d1c4-48e3-a355-a99aa9fa4bb4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:29:46.236423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: warnings.warn(msg) Aug 01 15:29:46.236423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: Aug 01 15:29:46.237614 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd6a43-d1c4-48e3-a355-a99aa9fa4bb4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:46.282241 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 736/1498] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:29:46 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:29:46.286581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fea81-e63d-42ba-bb46-2226b64f8100 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:46.288129 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-401fea81-e63d-42ba-bb46-2226b64f8100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:46.350552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fea81-e63d-42ba-bb46-2226b64f8100 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:46.435090 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:46.447182 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b86e18-4195-4162-a6c2-c70f2e935d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:46.448419 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b86e18-4195-4162-a6c2-c70f2e935d31 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:46.448419 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b86e18-4195-4162-a6c2-c70f2e935d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:46.448419 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b86e18-4195-4162-a6c2-c70f2e935d31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:46.449026 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b86e18-4195-4162-a6c2-c70f2e935d31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:46.450082 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: WARNING py.warnings [None req-97b86e18-4195-4162-a6c2-c70f2e935d31 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:29:46.450082 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: warnings.warn(msg) Aug 01 15:29:46.450082 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: Aug 01 15:29:46.450973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b86e18-4195-4162-a6c2-c70f2e935d31 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:46.463766 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 763/1499] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:29:46.467822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd6a43-d1c4-48e3-a355-a99aa9fa4bb4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:46.469404 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-96fd6a43-d1c4-48e3-a355-a99aa9fa4bb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:46.481456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd6a43-d1c4-48e3-a355-a99aa9fa4bb4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:46.507635 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:46.578916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61c304-9c29-448d-9fed-e56cee6e2211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:46.579471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61c304-9c29-448d-9fed-e56cee6e2211 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:46.579753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61c304-9c29-448d-9fed-e56cee6e2211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:46.580028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61c304-9c29-448d-9fed-e56cee6e2211 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:46.580715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61c304-9c29-448d-9fed-e56cee6e2211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:46.581738 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: WARNING py.warnings [None req-fe61c304-9c29-448d-9fed-e56cee6e2211 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:29:46.581738 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: warnings.warn(msg) Aug 01 15:29:46.581738 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: Aug 01 15:29:46.582655 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61c304-9c29-448d-9fed-e56cee6e2211 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:46.594915 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 737/1500] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:29:46.599113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b86e18-4195-4162-a6c2-c70f2e935d31 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:46.600659 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-97b86e18-4195-4162-a6c2-c70f2e935d31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:46.635288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b86e18-4195-4162-a6c2-c70f2e935d31 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:46.729343 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328bf66-af94-4aec-a9b9-304fa65140cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:46.730046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328bf66-af94-4aec-a9b9-304fa65140cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:46.730359 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328bf66-af94-4aec-a9b9-304fa65140cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:46.730634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328bf66-af94-4aec-a9b9-304fa65140cc admin admin] PATH_INFO: `/v3/projects/1d76eaf6a70341faaae38ef85af97242/users/6b1a142b1f744c95aa90dd57b09c4594/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:46.753727 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328bf66-af94-4aec-a9b9-304fa65140cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d76eaf6a70341faaae38ef85af97242, user_id=6b1a142b1f744c95aa90dd57b09c4594, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:46.754802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: WARNING py.warnings [None req-5328bf66-af94-4aec-a9b9-304fa65140cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:29:46.754802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: warnings.warn(msg) Aug 01 15:29:46.754802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: Aug 01 15:29:46.756780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328bf66-af94-4aec-a9b9-304fa65140cc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:46.786262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 764/1501] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:29:46 2022] PUT /identity/v3/projects/1d76eaf6a70341faaae38ef85af97242/users/6b1a142b1f744c95aa90dd57b09c4594/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:46.799702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff764fe8-b0e5-47ff-95ce-e3feedf6292e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:46.800167 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff764fe8-b0e5-47ff-95ce-e3feedf6292e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:46.800418 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff764fe8-b0e5-47ff-95ce-e3feedf6292e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:46.858644 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ff764fe8-b0e5-47ff-95ce-e3feedf6292e None None] MFA Rules not processed for user `6b1a142b1f744c95aa90dd57b09c4594`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:46.859747 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ff764fe8-b0e5-47ff-95ce-e3feedf6292e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:46.939724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 738/1502] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:29:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:46.952981 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f539f-92d9-472f-8ed4-4ea7ce22da6a None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:46.953267 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f539f-92d9-472f-8ed4-4ea7ce22da6a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:46.953545 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f539f-92d9-472f-8ed4-4ea7ce22da6a None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:46.955180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 765/1503] 10.222.0.83 () {54 vars in 964 bytes} [Mon Aug 1 15:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:29:46.960253 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be05974-b01c-42fc-a356-c4668365dc45 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:46.960543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be05974-b01c-42fc-a356-c4668365dc45 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:46.960820 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be05974-b01c-42fc-a356-c4668365dc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:46.992621 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:47.037249 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-9be05974-b01c-42fc-a356-c4668365dc45 None None] MFA Rules not processed for user `f54b570990794801a5e45918845569b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:47.038602 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9be05974-b01c-42fc-a356-c4668365dc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:47.116078 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 739/1504] 10.222.0.83 () {58 vars in 1061 bytes} [Mon Aug 1 15:29:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:47.124220 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f539f-92d9-472f-8ed4-4ea7ce22da6a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:47.125796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9d7f539f-92d9-472f-8ed4-4ea7ce22da6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:47.182649 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:47.238397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f539f-92d9-472f-8ed4-4ea7ce22da6a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:47.317123 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dad24f-c9c2-4665-95f1-3eef2653b12e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:47.317741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dad24f-c9c2-4665-95f1-3eef2653b12e service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:47.318050 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dad24f-c9c2-4665-95f1-3eef2653b12e service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:47.318323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dad24f-c9c2-4665-95f1-3eef2653b12e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:47.320066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f9dad24f-c9c2-4665-95f1-3eef2653b12e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:47.417726 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dad24f-c9c2-4665-95f1-3eef2653b12e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:47.419100 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dad24f-c9c2-4665-95f1-3eef2653b12e service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:47.422569 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:47.437542 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 766/1505] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:29:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:47.485652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53198202-4472-428f-a912-8f1ff5c0ea41 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:47.486360 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53198202-4472-428f-a912-8f1ff5c0ea41 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:47.486360 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53198202-4472-428f-a912-8f1ff5c0ea41 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:47.487659 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 740/1506] 10.222.0.83 () {52 vars in 830 bytes} [Mon Aug 1 15:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:29:47.491768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dad24f-c9c2-4665-95f1-3eef2653b12e service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:47.493007 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f9dad24f-c9c2-4665-95f1-3eef2653b12e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:47.593185 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dad24f-c9c2-4665-95f1-3eef2653b12e service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:47.690658 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7ebfe-61f9-4a4e-8885-f361e828d8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:47.691341 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7ebfe-61f9-4a4e-8885-f361e828d8c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:47.691460 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7ebfe-61f9-4a4e-8885-f361e828d8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:47.691708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7ebfe-61f9-4a4e-8885-f361e828d8c7 admin admin] PATH_INFO: `/v3/users/6b1a142b1f744c95aa90dd57b09c4594` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:47.705022 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a7ebfe-61f9-4a4e-8885-f361e828d8c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b1a142b1f744c95aa90dd57b09c4594)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:47.706068 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: WARNING py.warnings [None req-30a7ebfe-61f9-4a4e-8885-f361e828d8c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:29:47.706068 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: warnings.warn(msg) Aug 01 15:29:47.706068 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: Aug 01 15:29:47.707273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a7ebfe-61f9-4a4e-8885-f361e828d8c7 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:47.716693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:47.774660 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-30a7ebfe-61f9-4a4e-8885-f361e828d8c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b1a142b1f744c95aa90dd57b09c4594'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:47.775368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-30a7ebfe-61f9-4a4e-8885-f361e828d8c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b1a142b1f744c95aa90dd57b09c4594'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:47.786113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-30a7ebfe-61f9-4a4e-8885-f361e828d8c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b1a142b1f744c95aa90dd57b09c4594'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:47.809941 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-30a7ebfe-61f9-4a4e-8885-f361e828d8c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b1a142b1f744c95aa90dd57b09c4594'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:47.824761 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-30a7ebfe-61f9-4a4e-8885-f361e828d8c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b1a142b1f744c95aa90dd57b09c4594'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:47.830603 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 767/1507] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:29:47 2022] DELETE /identity/v3/users/6b1a142b1f744c95aa90dd57b09c4594 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:47.834922 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2235060-b320-4856-85d5-2563940422ce None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:47.835212 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2235060-b320-4856-85d5-2563940422ce None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:47.835493 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2235060-b320-4856-85d5-2563940422ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:47.893680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f2235060-b320-4856-85d5-2563940422ce None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:47.894959 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f2235060-b320-4856-85d5-2563940422ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:47.972689 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:47.986270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 741/1508] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:29:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:47.995889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7ebfe-61f9-4a4e-8885-f361e828d8c7 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:47.997144 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-30a7ebfe-61f9-4a4e-8885-f361e828d8c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:48.098538 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7ebfe-61f9-4a4e-8885-f361e828d8c7 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:48.153298 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:48.174885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c038a-d431-4f68-94b1-7f4fae33355b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:48.175509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c038a-d431-4f68-94b1-7f4fae33355b service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:48.175790 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c038a-d431-4f68-94b1-7f4fae33355b service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:48.176062 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c038a-d431-4f68-94b1-7f4fae33355b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:48.177680 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9b1c038a-d431-4f68-94b1-7f4fae33355b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:48.285853 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c038a-d431-4f68-94b1-7f4fae33355b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:48.287251 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c038a-d431-4f68-94b1-7f4fae33355b service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:48.305825 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 768/1509] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:29:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:48.647925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2235060-b320-4856-85d5-2563940422ce None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:48.649238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f2235060-b320-4856-85d5-2563940422ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:48.661795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2235060-b320-4856-85d5-2563940422ce None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:48.761100 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717d279-10f1-4d6c-b133-330ecb05d561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:48.761732 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717d279-10f1-4d6c-b133-330ecb05d561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:48.762141 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717d279-10f1-4d6c-b133-330ecb05d561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:48.762348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717d279-10f1-4d6c-b133-330ecb05d561 admin admin] PATH_INFO: `/v3/projects/1d76eaf6a70341faaae38ef85af97242` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:48.773137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717d279-10f1-4d6c-b133-330ecb05d561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d76eaf6a70341faaae38ef85af97242)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:48.774209 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: WARNING py.warnings [None req-a717d279-10f1-4d6c-b133-330ecb05d561 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:29:48.774209 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: warnings.warn(msg) Aug 01 15:29:48.774209 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: Aug 01 15:29:48.775412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717d279-10f1-4d6c-b133-330ecb05d561 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:48.834440 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a717d279-10f1-4d6c-b133-330ecb05d561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d76eaf6a70341faaae38ef85af97242', 'request_id': 'req-a717d279-10f1-4d6c-b133-330ecb05d561'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:48.847995 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a717d279-10f1-4d6c-b133-330ecb05d561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d76eaf6a70341faaae38ef85af97242', 'request_id': 'req-a717d279-10f1-4d6c-b133-330ecb05d561'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:48.854025 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a717d279-10f1-4d6c-b133-330ecb05d561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d76eaf6a70341faaae38ef85af97242', 'request_id': 'req-a717d279-10f1-4d6c-b133-330ecb05d561'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:48.880214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a717d279-10f1-4d6c-b133-330ecb05d561 admin admin] The token cache is being invalidate because project 1d76eaf6a70341faaae38ef85af97242 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:29:48.880502 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a717d279-10f1-4d6c-b133-330ecb05d561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:48.881025 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a717d279-10f1-4d6c-b133-330ecb05d561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:48.882413 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 742/1510] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:29:48 2022] DELETE /identity/v3/projects/1d76eaf6a70341faaae38ef85af97242 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:58.303589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=62 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:58.878336 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:37.278449 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82200b-bf5c-40d6-81ca-c4795868fada None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:37.279165 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82200b-bf5c-40d6-81ca-c4795868fada None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:37.279165 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82200b-bf5c-40d6-81ca-c4795868fada None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:37.281141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 769/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:31:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:31:37.286801 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78d461-7cc1-4fb9-a595-65910c580fe3 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:37.287737 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78d461-7cc1-4fb9-a595-65910c580fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:37.287737 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78d461-7cc1-4fb9-a595-65910c580fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:37.383419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-fd78d461-7cc1-4fb9-a595-65910c580fe3 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:37.384802 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-fd78d461-7cc1-4fb9-a595-65910c580fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:37.509970 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 743/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:31:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:37.670715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82200b-bf5c-40d6-81ca-c4795868fada None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:37.673107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2f82200b-bf5c-40d6-81ca-c4795868fada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:37.801013 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82200b-bf5c-40d6-81ca-c4795868fada None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:37.806629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ef416-a932-476c-ac2a-ce5b16cabdfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:37.807247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ef416-a932-476c-ac2a-ce5b16cabdfb service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:37.807531 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ef416-a932-476c-ac2a-ce5b16cabdfb service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:37.807804 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ef416-a932-476c-ac2a-ce5b16cabdfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:37.816740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ef416-a932-476c-ac2a-ce5b16cabdfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:37.818362 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ef416-a932-476c-ac2a-ce5b16cabdfb service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:37.828480 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 770/1513] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:38.691438 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a4375-eb2f-4e51-a30b-e0ffd83f44e3 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:38.692000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a4375-eb2f-4e51-a30b-e0ffd83f44e3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:38.692000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a4375-eb2f-4e51-a30b-e0ffd83f44e3 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:38.693412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 744/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:31:38.698675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432bd86-2166-494a-9cf6-859bec554bbb None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:38.699184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432bd86-2166-494a-9cf6-859bec554bbb None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:38.699278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432bd86-2166-494a-9cf6-859bec554bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:38.722308 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-9432bd86-2166-494a-9cf6-859bec554bbb None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:38.723548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9432bd86-2166-494a-9cf6-859bec554bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:38.731872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 771/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:38.884973 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a4375-eb2f-4e51-a30b-e0ffd83f44e3 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:38.900258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a4375-eb2f-4e51-a30b-e0ffd83f44e3 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:38.906551 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a92f8-b11a-43a2-8513-55ff69ab6b13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:38.907306 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a92f8-b11a-43a2-8513-55ff69ab6b13 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:38.907688 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a92f8-b11a-43a2-8513-55ff69ab6b13 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:38.908073 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a92f8-b11a-43a2-8513-55ff69ab6b13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:38.917734 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a92f8-b11a-43a2-8513-55ff69ab6b13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:38.919396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a92f8-b11a-43a2-8513-55ff69ab6b13 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:38.929925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 745/1516] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:41.414160 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac15bf-a79d-44b2-923e-ce4d22c85884 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:41.414856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac15bf-a79d-44b2-923e-ce4d22c85884 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:41.414856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac15bf-a79d-44b2-923e-ce4d22c85884 None None] PATH_INFO: `/v3` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:41.416230 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 772/1517] 10.222.0.46 () {54 vars in 925 bytes} [Mon Aug 1 15:31:41 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:31:41.423760 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93175a-cb5c-4dbb-b62c-73855a49ae67 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:41.424532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93175a-cb5c-4dbb-b62c-73855a49ae67 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:41.424532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93175a-cb5c-4dbb-b62c-73855a49ae67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:41.492301 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-de93175a-cb5c-4dbb-b62c-73855a49ae67 None None] MFA Rules not processed for user `ef57a9394b114eb3ab499133b5d40781`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:41.493534 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-de93175a-cb5c-4dbb-b62c-73855a49ae67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:41.511367 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:41.571237 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 746/1518] 10.222.0.46 () {58 vars in 1015 bytes} [Mon Aug 1 15:31:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:45.954373 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71442d50-c5d8-45ba-9048-7753cff48332 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:45.954927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71442d50-c5d8-45ba-9048-7753cff48332 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:45.954927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71442d50-c5d8-45ba-9048-7753cff48332 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:45.956453 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 773/1519] 10.222.0.83 () {54 vars in 961 bytes} [Mon Aug 1 15:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:31:45.961693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b1f2f-429a-4498-bab0-928e68804e5e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:45.962221 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b1f2f-429a-4498-bab0-928e68804e5e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:45.962331 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b1f2f-429a-4498-bab0-928e68804e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:45.982814 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-8b0b1f2f-429a-4498-bab0-928e68804e5e None None] MFA Rules not processed for user `ef57a9394b114eb3ab499133b5d40781`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:45.984567 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8b0b1f2f-429a-4498-bab0-928e68804e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:45.996489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 747/1520] 10.222.0.83 () {58 vars in 1058 bytes} [Mon Aug 1 15:31:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:46.003893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71442d50-c5d8-45ba-9048-7753cff48332 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:46.018090 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71442d50-c5d8-45ba-9048-7753cff48332 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:46.030458 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efab3b0-c797-470b-8820-ee0eeba0630f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:46.031126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efab3b0-c797-470b-8820-ee0eeba0630f service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:46.031406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efab3b0-c797-470b-8820-ee0eeba0630f service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:46.032063 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efab3b0-c797-470b-8820-ee0eeba0630f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:46.042451 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efab3b0-c797-470b-8820-ee0eeba0630f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:46.044055 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efab3b0-c797-470b-8820-ee0eeba0630f service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:46.056289 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 774/1521] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:50.207369 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2781c-154d-4107-acb5-d9f2d14583ce None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:50.207945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2781c-154d-4107-acb5-d9f2d14583ce None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:50.207945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2781c-154d-4107-acb5-d9f2d14583ce None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:50.209287 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 748/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:31:50.214138 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667f458-cbd3-4d22-8e75-324b9a7a6428 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:50.214547 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667f458-cbd3-4d22-8e75-324b9a7a6428 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:50.214722 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667f458-cbd3-4d22-8e75-324b9a7a6428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:50.234303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a667f458-cbd3-4d22-8e75-324b9a7a6428 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:50.235511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a667f458-cbd3-4d22-8e75-324b9a7a6428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:50.243323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 775/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:50.397915 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2781c-154d-4107-acb5-d9f2d14583ce None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:50.412247 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2781c-154d-4107-acb5-d9f2d14583ce None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:50.417713 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd759b0-6db1-4f33-8ca1-6ec26dd6f8b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:50.418361 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd759b0-6db1-4f33-8ca1-6ec26dd6f8b6 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:50.418646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd759b0-6db1-4f33-8ca1-6ec26dd6f8b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:50.418926 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd759b0-6db1-4f33-8ca1-6ec26dd6f8b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:50.427519 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd759b0-6db1-4f33-8ca1-6ec26dd6f8b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:50.429126 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd759b0-6db1-4f33-8ca1-6ec26dd6f8b6 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:50.439225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 749/1524] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:51.278911 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43635fb7-9fa8-4e5e-85b8-d525f4dbd80d None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:51.279511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43635fb7-9fa8-4e5e-85b8-d525f4dbd80d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:51.279511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43635fb7-9fa8-4e5e-85b8-d525f4dbd80d None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:51.280847 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 776/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:31:51.285718 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f471911-d126-4eef-83dc-2d71cff78bb9 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:51.286187 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f471911-d126-4eef-83dc-2d71cff78bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:51.286317 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f471911-d126-4eef-83dc-2d71cff78bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:51.306011 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-8f471911-d126-4eef-83dc-2d71cff78bb9 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:51.307220 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8f471911-d126-4eef-83dc-2d71cff78bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:51.315814 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 750/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:51.467526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43635fb7-9fa8-4e5e-85b8-d525f4dbd80d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:51.482432 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43635fb7-9fa8-4e5e-85b8-d525f4dbd80d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:51.488592 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ea6ad-f62b-4a6f-96be-33cdc62003bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:51.489196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ea6ad-f62b-4a6f-96be-33cdc62003bd service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:51.489478 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ea6ad-f62b-4a6f-96be-33cdc62003bd service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:51.489752 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ea6ad-f62b-4a6f-96be-33cdc62003bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:51.497976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ea6ad-f62b-4a6f-96be-33cdc62003bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:51.499523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ea6ad-f62b-4a6f-96be-33cdc62003bd service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:51.508884 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 777/1527] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:55.977786 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6e43b-aa82-4a9a-bd64-5526c8cf935b None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:55.978414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6e43b-aa82-4a9a-bd64-5526c8cf935b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:55.978414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6e43b-aa82-4a9a-bd64-5526c8cf935b None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:55.979747 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 751/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:31:55.985046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bb93f-6a3c-4e58-8225-2d4b6f3a17ca None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:55.986195 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bb93f-6a3c-4e58-8225-2d4b6f3a17ca None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:55.986476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bb93f-6a3c-4e58-8225-2d4b6f3a17ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:56.005742 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-462bb93f-6a3c-4e58-8225-2d4b6f3a17ca None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:56.006985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-462bb93f-6a3c-4e58-8225-2d4b6f3a17ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:56.014930 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 778/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:56.092427 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6e43b-aa82-4a9a-bd64-5526c8cf935b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:56.094177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-94c6e43b-aa82-4a9a-bd64-5526c8cf935b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:56.174057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6e43b-aa82-4a9a-bd64-5526c8cf935b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:56.177680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f2a6c-7630-45cf-accc-7c250a6b08a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:56.178337 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f2a6c-7630-45cf-accc-7c250a6b08a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:56.178621 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f2a6c-7630-45cf-accc-7c250a6b08a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:56.178897 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f2a6c-7630-45cf-accc-7c250a6b08a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:56.187993 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f2a6c-7630-45cf-accc-7c250a6b08a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:56.189471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f2a6c-7630-45cf-accc-7c250a6b08a7 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:56.198752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 752/1530] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:58.264258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9e338-eb09-4c45-9691-6f26472836b0 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:58.264839 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9e338-eb09-4c45-9691-6f26472836b0 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:58.264839 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9e338-eb09-4c45-9691-6f26472836b0 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:58.266861 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 779/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:31:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:31:58.272690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84605052-b2df-410e-a302-cd7c8414b373 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:58.273237 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84605052-b2df-410e-a302-cd7c8414b373 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:58.273343 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84605052-b2df-410e-a302-cd7c8414b373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:58.304641 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-84605052-b2df-410e-a302-cd7c8414b373 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:58.305972 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-84605052-b2df-410e-a302-cd7c8414b373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:58.314575 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 753/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:31:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:58.356434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9e338-eb09-4c45-9691-6f26472836b0 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:58.358363 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3bc9e338-eb09-4c45-9691-6f26472836b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:58.369380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=75 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:58.377392 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9e338-eb09-4c45-9691-6f26472836b0 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:58.381326 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c155652-fa02-42fb-a158-b6cb855434d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:58.382329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c155652-fa02-42fb-a158-b6cb855434d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:58.382815 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c155652-fa02-42fb-a158-b6cb855434d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:58.383254 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c155652-fa02-42fb-a158-b6cb855434d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:58.400618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c155652-fa02-42fb-a158-b6cb855434d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:58.403320 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c155652-fa02-42fb-a158-b6cb855434d7 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:58.421411 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 780/1533] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:02.466824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc21c71-7ed8-4e02-936b-4f1784ac1c93 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:02.467394 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc21c71-7ed8-4e02-936b-4f1784ac1c93 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:02.467394 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc21c71-7ed8-4e02-936b-4f1784ac1c93 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:02.468760 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 754/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:32:02.474756 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67930851-a131-43ad-a637-78074e24c05e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:02.475206 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67930851-a131-43ad-a637-78074e24c05e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:02.475326 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67930851-a131-43ad-a637-78074e24c05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:02.495249 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-67930851-a131-43ad-a637-78074e24c05e None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:02.496480 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-67930851-a131-43ad-a637-78074e24c05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:02.505296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 781/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:02.816912 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc21c71-7ed8-4e02-936b-4f1784ac1c93 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:02.818635 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3fc21c71-7ed8-4e02-936b-4f1784ac1c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:02.885551 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:02.901838 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc21c71-7ed8-4e02-936b-4f1784ac1c93 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:02.905460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a2b40-7ccd-40f4-8feb-a4e79990af08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:02.906123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a2b40-7ccd-40f4-8feb-a4e79990af08 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:02.906409 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a2b40-7ccd-40f4-8feb-a4e79990af08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:02.906685 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a2b40-7ccd-40f4-8feb-a4e79990af08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:02.917322 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a2b40-7ccd-40f4-8feb-a4e79990af08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:02.918856 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a2b40-7ccd-40f4-8feb-a4e79990af08 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:02.931210 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 755/1536] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:05.022481 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121863d-e830-4bca-87ce-69b23ccdec49 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:05.023050 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121863d-e830-4bca-87ce-69b23ccdec49 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:05.023050 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121863d-e830-4bca-87ce-69b23ccdec49 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:05.024414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 782/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:32:05.030168 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348de01a-9de0-4f22-acf1-6f9d5bca73c3 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:05.030683 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348de01a-9de0-4f22-acf1-6f9d5bca73c3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:05.030788 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348de01a-9de0-4f22-acf1-6f9d5bca73c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:05.050768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-348de01a-9de0-4f22-acf1-6f9d5bca73c3 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:05.052014 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-348de01a-9de0-4f22-acf1-6f9d5bca73c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:05.060066 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 756/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:05.377862 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121863d-e830-4bca-87ce-69b23ccdec49 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:05.379698 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0121863d-e830-4bca-87ce-69b23ccdec49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:05.392905 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121863d-e830-4bca-87ce-69b23ccdec49 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:05.396457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ad1f7-4037-485a-a34b-4c7dc54d0b39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:05.397069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ad1f7-4037-485a-a34b-4c7dc54d0b39 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:05.397351 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ad1f7-4037-485a-a34b-4c7dc54d0b39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:05.397624 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ad1f7-4037-485a-a34b-4c7dc54d0b39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:05.407308 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ad1f7-4037-485a-a34b-4c7dc54d0b39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:05.408739 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ad1f7-4037-485a-a34b-4c7dc54d0b39 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:05.418318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 783/1539] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:15.045161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=11 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:15.414039 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=14 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:40.357404 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fed42e-6bb5-4d65-8d4e-69927deca5d7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:40.359050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fed42e-6bb5-4d65-8d4e-69927deca5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:40.359050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fed42e-6bb5-4d65-8d4e-69927deca5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:40.388271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a1fed42e-6bb5-4d65-8d4e-69927deca5d7 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:40.389569 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a1fed42e-6bb5-4d65-8d4e-69927deca5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:40.400764 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 757/1540] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:33:40.411911 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faae9f-c0d6-4120-bc56-95fe99b3ee61 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:40.413107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faae9f-c0d6-4120-bc56-95fe99b3ee61 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:40.413640 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faae9f-c0d6-4120-bc56-95fe99b3ee61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:40.448816 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-57faae9f-c0d6-4120-bc56-95fe99b3ee61 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:40.450484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-57faae9f-c0d6-4120-bc56-95fe99b3ee61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:40.464257 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301cc6d-64df-4c17-89ec-5115a06c2e25 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:40.464595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301cc6d-64df-4c17-89ec-5115a06c2e25 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:40.464813 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301cc6d-64df-4c17-89ec-5115a06c2e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:40.469035 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 784/1541] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:40.479644 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57faae9f-c0d6-4120-bc56-95fe99b3ee61 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:40.500642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57faae9f-c0d6-4120-bc56-95fe99b3ee61 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:40.500880 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6301cc6d-64df-4c17-89ec-5115a06c2e25 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:40.504163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6301cc6d-64df-4c17-89ec-5115a06c2e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:40.516650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af8a40-4e54-4a6a-82f6-fc8aab2999ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:40.518081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af8a40-4e54-4a6a-82f6-fc8aab2999ca admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:40.518837 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af8a40-4e54-4a6a-82f6-fc8aab2999ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:40.520464 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af8a40-4e54-4a6a-82f6-fc8aab2999ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:40.520664 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 758/1542] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:40.523024 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af8a40-4e54-4a6a-82f6-fc8aab2999ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:33:40.523796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af8a40-4e54-4a6a-82f6-fc8aab2999ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:40.525072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: WARNING py.warnings [None req-22af8a40-4e54-4a6a-82f6-fc8aab2999ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:33:40.525072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: warnings.warn(msg) Aug 01 15:33:40.525072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: Aug 01 15:33:40.529918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af8a40-4e54-4a6a-82f6-fc8aab2999ca admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:40.532223 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76755167-536d-409e-aa7a-361ea09e13a3 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:40.532565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76755167-536d-409e-aa7a-361ea09e13a3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:40.534403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76755167-536d-409e-aa7a-361ea09e13a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:40.554522 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 785/1543] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:33:40.564882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af8a40-4e54-4a6a-82f6-fc8aab2999ca admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:40.584325 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-76755167-536d-409e-aa7a-361ea09e13a3 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:40.587385 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-76755167-536d-409e-aa7a-361ea09e13a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:40.589707 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af8a40-4e54-4a6a-82f6-fc8aab2999ca admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:40.607992 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4247aa-19a0-41c5-a405-f94e99b5a147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:40.609151 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4247aa-19a0-41c5-a405-f94e99b5a147 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:40.609665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4247aa-19a0-41c5-a405-f94e99b5a147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:40.610254 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4247aa-19a0-41c5-a405-f94e99b5a147 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:40.611393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4247aa-19a0-41c5-a405-f94e99b5a147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:40.611778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 759/1544] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:40.620845 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76755167-536d-409e-aa7a-361ea09e13a3 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:40.622541 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4247aa-19a0-41c5-a405-f94e99b5a147 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:40.634066 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76755167-536d-409e-aa7a-361ea09e13a3 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:40.659575 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd6f31-32f8-4d8b-9b0e-79edfdbbb372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:40.660290 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd6f31-32f8-4d8b-9b0e-79edfdbbb372 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:40.660579 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd6f31-32f8-4d8b-9b0e-79edfdbbb372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:40.660859 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd6f31-32f8-4d8b-9b0e-79edfdbbb372 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:40.661592 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd6f31-32f8-4d8b-9b0e-79edfdbbb372 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:33:40.661923 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd6f31-32f8-4d8b-9b0e-79edfdbbb372 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:40.663582 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd6f31-32f8-4d8b-9b0e-79edfdbbb372 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:40.696192 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 760/1545] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:33:40.697944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985da089-250b-4e86-80f0-93e6c9d6e208 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:40.698516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985da089-250b-4e86-80f0-93e6c9d6e208 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:40.698654 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985da089-250b-4e86-80f0-93e6c9d6e208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:40.716177 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 786/1546] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:33:40 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:33:40.717738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e09500-d0c3-4fc0-af8b-066fc03cd024 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:40.718061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e09500-d0c3-4fc0-af8b-066fc03cd024 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:40.718339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e09500-d0c3-4fc0-af8b-066fc03cd024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:40.732370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-985da089-250b-4e86-80f0-93e6c9d6e208 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:40.734565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-985da089-250b-4e86-80f0-93e6c9d6e208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:40.748450 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f9e09500-d0c3-4fc0-af8b-066fc03cd024 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:40.751833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f9e09500-d0c3-4fc0-af8b-066fc03cd024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:40.829956 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 761/1547] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:40.831341 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6d674-c3ec-4e5f-8ed6-7906ce31d547 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:40.831628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6d674-c3ec-4e5f-8ed6-7906ce31d547 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:40.831904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6d674-c3ec-4e5f-8ed6-7906ce31d547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:40.841365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 787/1548] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:40.843301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1673e5-1471-4fdd-91ae-8a002e637288 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:40.843828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1673e5-1471-4fdd-91ae-8a002e637288 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:40.844343 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1673e5-1471-4fdd-91ae-8a002e637288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:40.860498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d8b6d674-c3ec-4e5f-8ed6-7906ce31d547 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:40.861623 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d8b6d674-c3ec-4e5f-8ed6-7906ce31d547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:40.867843 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-fb1673e5-1471-4fdd-91ae-8a002e637288 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:40.869324 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fb1673e5-1471-4fdd-91ae-8a002e637288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:40.877251 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 762/1549] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:40.877251 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6d674-c3ec-4e5f-8ed6-7906ce31d547 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:40.884075 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6d674-c3ec-4e5f-8ed6-7906ce31d547 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:40.890816 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29181d1-8497-4b9c-b187-303f88650d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:40.891174 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 788/1550] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:40.892044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29181d1-8497-4b9c-b187-303f88650d40 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:40.892180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1673e5-1471-4fdd-91ae-8a002e637288 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:40.892811 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29181d1-8497-4b9c-b187-303f88650d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:40.893335 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29181d1-8497-4b9c-b187-303f88650d40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:40.894289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29181d1-8497-4b9c-b187-303f88650d40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:40.895605 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: WARNING py.warnings [None req-d29181d1-8497-4b9c-b187-303f88650d40 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:33:40.895605 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: warnings.warn(msg) Aug 01 15:33:40.895605 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: Aug 01 15:33:40.897353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29181d1-8497-4b9c-b187-303f88650d40 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:40.904848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1673e5-1471-4fdd-91ae-8a002e637288 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:40.912269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2e80f-1457-43f6-b069-61b3c4bb2173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:40.912920 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2e80f-1457-43f6-b069-61b3c4bb2173 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:40.913295 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2e80f-1457-43f6-b069-61b3c4bb2173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:40.913546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2e80f-1457-43f6-b069-61b3c4bb2173 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:40.914294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2e80f-1457-43f6-b069-61b3c4bb2173 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:40.915717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: WARNING py.warnings [None req-d3c2e80f-1457-43f6-b069-61b3c4bb2173 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:33:40.915717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: warnings.warn(msg) Aug 01 15:33:40.915717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: Aug 01 15:33:40.916669 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2e80f-1457-43f6-b069-61b3c4bb2173 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:40.943194 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 763/1551] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:33:40 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:40.943194 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee620df7-ae03-44cc-ab6c-6e080c3d2276 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:40.943194 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee620df7-ae03-44cc-ab6c-6e080c3d2276 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:40.943194 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee620df7-ae03-44cc-ab6c-6e080c3d2276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:40.954098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 789/1552] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:33:40 2022] POST /identity/v3/users => generated 293 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:40.954098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97728409-b420-4300-a5f4-77ebcb579356 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:40.954098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97728409-b420-4300-a5f4-77ebcb579356 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:40.954098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97728409-b420-4300-a5f4-77ebcb579356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:40.974812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ee620df7-ae03-44cc-ab6c-6e080c3d2276 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:40.975151 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-97728409-b420-4300-a5f4-77ebcb579356 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:40.976070 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ee620df7-ae03-44cc-ab6c-6e080c3d2276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:40.976352 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-97728409-b420-4300-a5f4-77ebcb579356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:41.066620 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 790/1553] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:41.066620 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979e347-66f8-46b4-8ad6-cfe9d58e9a6f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.066620 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979e347-66f8-46b4-8ad6-cfe9d58e9a6f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.066620 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979e347-66f8-46b4-8ad6-cfe9d58e9a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.067417 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 764/1554] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:41.068213 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b9020-a876-43b2-89d8-d07a71554b4d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.068510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b9020-a876-43b2-89d8-d07a71554b4d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.068788 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b9020-a876-43b2-89d8-d07a71554b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.090212 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2979e347-66f8-46b4-8ad6-cfe9d58e9a6f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:41.091431 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2979e347-66f8-46b4-8ad6-cfe9d58e9a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:41.091876 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f30b9020-a876-43b2-89d8-d07a71554b4d None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:41.093596 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f30b9020-a876-43b2-89d8-d07a71554b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:41.100019 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 791/1555] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:41.101833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a6909-3967-404f-ac3a-2591015b012e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.102014 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 765/1556] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:41.102204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a6909-3967-404f-ac3a-2591015b012e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.102461 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a6909-3967-404f-ac3a-2591015b012e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.102679 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b9020-a876-43b2-89d8-d07a71554b4d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.113311 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b9020-a876-43b2-89d8-d07a71554b4d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.117925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90611218-cb55-4f3a-9512-057da5fcacda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.118563 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90611218-cb55-4f3a-9512-057da5fcacda admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.118847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90611218-cb55-4f3a-9512-057da5fcacda admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.119232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90611218-cb55-4f3a-9512-057da5fcacda admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.119801 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90611218-cb55-4f3a-9512-057da5fcacda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:41.121862 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90611218-cb55-4f3a-9512-057da5fcacda admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:41.125342 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1d3a6909-3967-404f-ac3a-2591015b012e None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:41.126610 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1d3a6909-3967-404f-ac3a-2591015b012e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:41.138052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 792/1557] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:41.138920 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a6909-3967-404f-ac3a-2591015b012e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.150094 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a6909-3967-404f-ac3a-2591015b012e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.150835 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 766/1558] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:33:41 2022] POST /identity/v3/users => generated 294 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:41.152132 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc528522-8fa8-495f-99cf-c7802485d2f3 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.152419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc528522-8fa8-495f-99cf-c7802485d2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.152697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc528522-8fa8-495f-99cf-c7802485d2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.156318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ffad1-bf39-44be-ab49-a7e4022ce3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.156989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ffad1-bf39-44be-ab49-a7e4022ce3cf admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.157226 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ffad1-bf39-44be-ab49-a7e4022ce3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.157492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ffad1-bf39-44be-ab49-a7e4022ce3cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.158259 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ffad1-bf39-44be-ab49-a7e4022ce3cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:41.159845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ffad1-bf39-44be-ab49-a7e4022ce3cf admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:41.171701 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 793/1559] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:41.172578 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ffad1-bf39-44be-ab49-a7e4022ce3cf admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.175417 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-cc528522-8fa8-495f-99cf-c7802485d2f3 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:41.177030 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-cc528522-8fa8-495f-99cf-c7802485d2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:41.178348 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:41.187638 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ffad1-bf39-44be-ab49-a7e4022ce3cf admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.195566 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc72d2d-d42a-447c-a198-30ac79932a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.196193 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc72d2d-d42a-447c-a198-30ac79932a3b admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.196580 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc72d2d-d42a-447c-a198-30ac79932a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.196746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc72d2d-d42a-447c-a198-30ac79932a3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.197436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc72d2d-d42a-447c-a198-30ac79932a3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:33:41.197917 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc72d2d-d42a-447c-a198-30ac79932a3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:41.199404 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc72d2d-d42a-447c-a198-30ac79932a3b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:41.201959 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 767/1560] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:41.203571 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33683a3-dd56-439f-85b4-cc3d1d9cfb21 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.203924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33683a3-dd56-439f-85b4-cc3d1d9cfb21 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.204193 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33683a3-dd56-439f-85b4-cc3d1d9cfb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.215211 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 794/1561] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:33:41.216228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc72d2d-d42a-447c-a198-30ac79932a3b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.227640 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc72d2d-d42a-447c-a198-30ac79932a3b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.229661 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c33683a3-dd56-439f-85b4-cc3d1d9cfb21 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:41.230909 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c33683a3-dd56-439f-85b4-cc3d1d9cfb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:41.232425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4b0a4-d1b5-4563-8372-c4d1928bc98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.233055 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4b0a4-d1b5-4563-8372-c4d1928bc98c admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.233947 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4b0a4-d1b5-4563-8372-c4d1928bc98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.234216 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4b0a4-d1b5-4563-8372-c4d1928bc98c admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.234920 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e4b0a4-d1b5-4563-8372-c4d1928bc98c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:33:41.235230 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e4b0a4-d1b5-4563-8372-c4d1928bc98c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:41.236839 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e4b0a4-d1b5-4563-8372-c4d1928bc98c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:41.240518 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 768/1562] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:41.241364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33683a3-dd56-439f-85b4-cc3d1d9cfb21 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.246616 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:41.252119 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33683a3-dd56-439f-85b4-cc3d1d9cfb21 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.257725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 795/1563] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:33:41.258747 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4b0a4-d1b5-4563-8372-c4d1928bc98c admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.259074 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e113d9-a75c-491b-aa0a-c5ec56e07508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.259994 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e113d9-a75c-491b-aa0a-c5ec56e07508 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.260287 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e113d9-a75c-491b-aa0a-c5ec56e07508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.260697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e113d9-a75c-491b-aa0a-c5ec56e07508 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.261578 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e113d9-a75c-491b-aa0a-c5ec56e07508 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:33:41.261918 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e113d9-a75c-491b-aa0a-c5ec56e07508 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:41.263885 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e113d9-a75c-491b-aa0a-c5ec56e07508 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:41.270372 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4b0a4-d1b5-4563-8372-c4d1928bc98c admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.275720 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9f1b4-8043-4b34-8cd9-d45203c43657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.276376 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9f1b4-8043-4b34-8cd9-d45203c43657 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.276663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9f1b4-8043-4b34-8cd9-d45203c43657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.276944 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9f1b4-8043-4b34-8cd9-d45203c43657 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.277650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9f1b4-8043-4b34-8cd9-d45203c43657 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:33:41.277979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9f1b4-8043-4b34-8cd9-d45203c43657 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:41.279406 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 769/1564] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:33:41.279620 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9f1b4-8043-4b34-8cd9-d45203c43657 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:41.280892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e113d9-a75c-491b-aa0a-c5ec56e07508 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.292358 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e113d9-a75c-491b-aa0a-c5ec56e07508 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.294253 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 796/1565] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:33:41.295261 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9f1b4-8043-4b34-8cd9-d45203c43657 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.304214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ff82d-29ad-480d-81de-2f6882a653dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.305076 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ff82d-29ad-480d-81de-2f6882a653dd admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.305442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ff82d-29ad-480d-81de-2f6882a653dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.306346 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ff82d-29ad-480d-81de-2f6882a653dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.306647 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ff82d-29ad-480d-81de-2f6882a653dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:41.306860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9f1b4-8043-4b34-8cd9-d45203c43657 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.308670 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ff82d-29ad-480d-81de-2f6882a653dd admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:41.315971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a1e567-ba65-40ee-8c79-53c5aa8afffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.316958 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1e567-ba65-40ee-8c79-53c5aa8afffa admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.317440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1e567-ba65-40ee-8c79-53c5aa8afffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.317919 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1e567-ba65-40ee-8c79-53c5aa8afffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.319109 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a1e567-ba65-40ee-8c79-53c5aa8afffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:41.322034 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a1e567-ba65-40ee-8c79-53c5aa8afffa admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:41.328051 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 770/1566] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:41.328051 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec639f-5184-4711-9590-19ac43947e29 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.328051 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec639f-5184-4711-9590-19ac43947e29 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.328051 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec639f-5184-4711-9590-19ac43947e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.338296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 797/1567] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:41.338912 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a1e567-ba65-40ee-8c79-53c5aa8afffa admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.349601 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a6ec639f-5184-4711-9590-19ac43947e29 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:41.350861 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a6ec639f-5184-4711-9590-19ac43947e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:41.352296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a1e567-ba65-40ee-8c79-53c5aa8afffa admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.357122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac2503-44cf-4848-8a95-c62d4715d2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.357749 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac2503-44cf-4848-8a95-c62d4715d2d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.358057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac2503-44cf-4848-8a95-c62d4715d2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.358965 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac2503-44cf-4848-8a95-c62d4715d2d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.359625 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac2503-44cf-4848-8a95-c62d4715d2d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:41.361546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac2503-44cf-4848-8a95-c62d4715d2d6 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:41.363647 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 771/1568] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:41.366770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec639f-5184-4711-9590-19ac43947e29 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.376863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec639f-5184-4711-9590-19ac43947e29 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.383299 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cedcd4f-62e0-452c-b233-a8f3b3017bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.383986 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedcd4f-62e0-452c-b233-a8f3b3017bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.384274 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedcd4f-62e0-452c-b233-a8f3b3017bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.384622 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedcd4f-62e0-452c-b233-a8f3b3017bf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.385441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cedcd4f-62e0-452c-b233-a8f3b3017bf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:33:41.385823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cedcd4f-62e0-452c-b233-a8f3b3017bf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:41.387481 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cedcd4f-62e0-452c-b233-a8f3b3017bf0 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:41.393998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 798/1569] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:33:41 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:41.398006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac2503-44cf-4848-8a95-c62d4715d2d6 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.408578 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 772/1570] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:33:41.408578 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cedcd4f-62e0-452c-b233-a8f3b3017bf0 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.426011 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cedcd4f-62e0-452c-b233-a8f3b3017bf0 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.465162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac2503-44cf-4848-8a95-c62d4715d2d6 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.471476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea99a4-9ea7-49b9-8fe6-0f09fa875636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.473393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea99a4-9ea7-49b9-8fe6-0f09fa875636 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.473818 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea99a4-9ea7-49b9-8fe6-0f09fa875636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.474296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea99a4-9ea7-49b9-8fe6-0f09fa875636 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.475562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea99a4-9ea7-49b9-8fe6-0f09fa875636 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:41.477774 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea99a4-9ea7-49b9-8fe6-0f09fa875636 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:41.491431 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fc7a1-f74c-405f-b08a-6ef5623d0bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.492068 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fc7a1-f74c-405f-b08a-6ef5623d0bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.492354 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fc7a1-f74c-405f-b08a-6ef5623d0bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.492634 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fc7a1-f74c-405f-b08a-6ef5623d0bd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.493301 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672fc7a1-f74c-405f-b08a-6ef5623d0bd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:41.495299 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672fc7a1-f74c-405f-b08a-6ef5623d0bd4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:41.510916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 799/1571] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:33:41 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:41.512128 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea99a4-9ea7-49b9-8fe6-0f09fa875636 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.535040 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 773/1572] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:33:41 2022] POST /identity/v3/projects => generated 403 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:41.536192 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fc7a1-f74c-405f-b08a-6ef5623d0bd4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.585478 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea99a4-9ea7-49b9-8fe6-0f09fa875636 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.618091 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:41.620753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fc7a1-f74c-405f-b08a-6ef5623d0bd4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.625870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90459589-60d5-487a-8116-77c3c3917c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.626532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90459589-60d5-487a-8116-77c3c3917c77 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.626818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90459589-60d5-487a-8116-77c3c3917c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.627602 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90459589-60d5-487a-8116-77c3c3917c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.627994 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90459589-60d5-487a-8116-77c3c3917c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:41.629705 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90459589-60d5-487a-8116-77c3c3917c77 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:41.640654 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 774/1573] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:41.641491 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90459589-60d5-487a-8116-77c3c3917c77 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.652042 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90459589-60d5-487a-8116-77c3c3917c77 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.658082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013fa50-81e7-4d32-8845-ba0e6155cb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.658472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013fa50-81e7-4d32-8845-ba0e6155cb85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.658643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013fa50-81e7-4d32-8845-ba0e6155cb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.658922 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013fa50-81e7-4d32-8845-ba0e6155cb85 admin admin] PATH_INFO: `/v3/projects/3904f939fbfb43c280dd8e547f3d4604/users/3bd2843f2b2a4aaeb39b9dd10b32351a/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.675603 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013fa50-81e7-4d32-8845-ba0e6155cb85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3904f939fbfb43c280dd8e547f3d4604, user_id=3bd2843f2b2a4aaeb39b9dd10b32351a, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:41.676684 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: WARNING py.warnings [None req-e013fa50-81e7-4d32-8845-ba0e6155cb85 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:33:41.676684 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: warnings.warn(msg) Aug 01 15:33:41.676684 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: Aug 01 15:33:41.678508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811de11-3093-4699-81ec-86a5b68ae4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.678681 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013fa50-81e7-4d32-8845-ba0e6155cb85 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:41.679549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811de11-3093-4699-81ec-86a5b68ae4ee admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.679897 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811de11-3093-4699-81ec-86a5b68ae4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.680269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811de11-3093-4699-81ec-86a5b68ae4ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.681134 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811de11-3093-4699-81ec-86a5b68ae4ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:41.683589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811de11-3093-4699-81ec-86a5b68ae4ee admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:41.693374 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 775/1574] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:33:41 2022] PUT /identity/v3/projects/3904f939fbfb43c280dd8e547f3d4604/users/3bd2843f2b2a4aaeb39b9dd10b32351a/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:41.694432 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013fa50-81e7-4d32-8845-ba0e6155cb85 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.714335 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:41.723571 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 800/1575] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:33:41 2022] POST /identity/v3/projects => generated 377 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:41.724541 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811de11-3093-4699-81ec-86a5b68ae4ee admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.778064 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013fa50-81e7-4d32-8845-ba0e6155cb85 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.801273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811de11-3093-4699-81ec-86a5b68ae4ee admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.806238 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cfc2b-3851-4df8-9bcf-a5dfc17cf081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.806855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cfc2b-3851-4df8-9bcf-a5dfc17cf081 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.807318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cfc2b-3851-4df8-9bcf-a5dfc17cf081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.807478 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cfc2b-3851-4df8-9bcf-a5dfc17cf081 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.808066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12cfc2b-3851-4df8-9bcf-a5dfc17cf081 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:41.810049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12cfc2b-3851-4df8-9bcf-a5dfc17cf081 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:41.833266 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 801/1576] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:33:41 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:41.837173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cfc2b-3851-4df8-9bcf-a5dfc17cf081 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.840127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112cc8fa-4d45-4a93-bcbd-a6097b13b477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.840765 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cc8fa-4d45-4a93-bcbd-a6097b13b477 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.841051 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cc8fa-4d45-4a93-bcbd-a6097b13b477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.841328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cc8fa-4d45-4a93-bcbd-a6097b13b477 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.842123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112cc8fa-4d45-4a93-bcbd-a6097b13b477 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:33:41.842356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112cc8fa-4d45-4a93-bcbd-a6097b13b477 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:41.844575 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112cc8fa-4d45-4a93-bcbd-a6097b13b477 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:41.849801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cfc2b-3851-4df8-9bcf-a5dfc17cf081 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.854922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d345766-6292-43ac-b685-110c71a8c890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.855550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d345766-6292-43ac-b685-110c71a8c890 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.855840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d345766-6292-43ac-b685-110c71a8c890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.856187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d345766-6292-43ac-b685-110c71a8c890 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.856811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d345766-6292-43ac-b685-110c71a8c890 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:41.858470 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 776/1577] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:33:41.858862 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d345766-6292-43ac-b685-110c71a8c890 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:41.859232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112cc8fa-4d45-4a93-bcbd-a6097b13b477 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.874895 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112cc8fa-4d45-4a93-bcbd-a6097b13b477 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.880409 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad3f77-633f-49d6-9c2d-1472647cc07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.881061 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad3f77-633f-49d6-9c2d-1472647cc07d admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.881348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad3f77-633f-49d6-9c2d-1472647cc07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.881632 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad3f77-633f-49d6-9c2d-1472647cc07d admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.882432 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad3f77-633f-49d6-9c2d-1472647cc07d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:41.884385 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad3f77-633f-49d6-9c2d-1472647cc07d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:41.890789 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 802/1578] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:33:41 2022] POST /identity/v3/projects => generated 391 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:41.891615 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d345766-6292-43ac-b685-110c71a8c890 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.917040 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 777/1579] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:33:41 2022] POST /identity/v3/users => generated 290 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:41.917040 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad3f77-633f-49d6-9c2d-1472647cc07d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:41.925111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad3f77-633f-49d6-9c2d-1472647cc07d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.974557 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d345766-6292-43ac-b685-110c71a8c890 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:41.980078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475a57b-fc01-455a-a4fd-d26cbb91da2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.980817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475a57b-fc01-455a-a4fd-d26cbb91da2c admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.981171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475a57b-fc01-455a-a4fd-d26cbb91da2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.981509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475a57b-fc01-455a-a4fd-d26cbb91da2c admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:41.982509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475a57b-fc01-455a-a4fd-d26cbb91da2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:41.984585 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475a57b-fc01-455a-a4fd-d26cbb91da2c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:41.991105 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d8d6c-faa9-45f4-88a3-78cf9b46ed4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:41.991825 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d8d6c-faa9-45f4-88a3-78cf9b46ed4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:41.992100 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d8d6c-faa9-45f4-88a3-78cf9b46ed4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:41.992375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d8d6c-faa9-45f4-88a3-78cf9b46ed4a admin admin] PATH_INFO: `/v3/projects/e829698141514a009e2e769854fe66f0/users/7d1b6c388c7642a892c905d16f3b8d50/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.009408 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 803/1580] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:33:41 2022] POST /identity/v3/users => generated 302 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:42.011605 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d8d6c-faa9-45f4-88a3-78cf9b46ed4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e829698141514a009e2e769854fe66f0, user_id=7d1b6c388c7642a892c905d16f3b8d50, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:42.015081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973ef61-373e-4331-83d9-905ebe877665 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.015081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973ef61-373e-4331-83d9-905ebe877665 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.015352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d8d6c-faa9-45f4-88a3-78cf9b46ed4a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:42.015467 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973ef61-373e-4331-83d9-905ebe877665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.024962 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:42.032937 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 778/1581] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:33:41 2022] PUT /identity/v3/projects/e829698141514a009e2e769854fe66f0/users/7d1b6c388c7642a892c905d16f3b8d50/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:42.034853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d8d6c-faa9-45f4-88a3-78cf9b46ed4a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:42.057813 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-b973ef61-373e-4331-83d9-905ebe877665 None None] MFA Rules not processed for user `3bd2843f2b2a4aaeb39b9dd10b32351a`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:42.059589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b973ef61-373e-4331-83d9-905ebe877665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:42.073418 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:42.142836 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d8d6c-faa9-45f4-88a3-78cf9b46ed4a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:42.150042 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5bdb3a-c448-4d4d-91ca-88aa01bfbb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:42.150861 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bdb3a-c448-4d4d-91ca-88aa01bfbb2e admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.151260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bdb3a-c448-4d4d-91ca-88aa01bfbb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.151662 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bdb3a-c448-4d4d-91ca-88aa01bfbb2e admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.152550 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5bdb3a-c448-4d4d-91ca-88aa01bfbb2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:42.155365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5bdb3a-c448-4d4d-91ca-88aa01bfbb2e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:42.165112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 804/1582] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:42.165950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973ef61-373e-4331-83d9-905ebe877665 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:42.179533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973ef61-373e-4331-83d9-905ebe877665 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:42.184204 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 779/1583] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:33:42 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:42.185453 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5bdb3a-c448-4d4d-91ca-88aa01bfbb2e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:42.190349 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7addc2-5699-47b5-855b-e3793a18c925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:42.191006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7addc2-5699-47b5-855b-e3793a18c925 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.191501 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7addc2-5699-47b5-855b-e3793a18c925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.191621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7addc2-5699-47b5-855b-e3793a18c925 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.192346 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7addc2-5699-47b5-855b-e3793a18c925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:42.194020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7addc2-5699-47b5-855b-e3793a18c925 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:42.210007 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 805/1584] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:42.210007 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7addc2-5699-47b5-855b-e3793a18c925 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:42.210399 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5bdb3a-c448-4d4d-91ca-88aa01bfbb2e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:42.216168 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91369e2-5fdb-4d15-abf8-4979e30ef67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:42.216909 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91369e2-5fdb-4d15-abf8-4979e30ef67f admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.217690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91369e2-5fdb-4d15-abf8-4979e30ef67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.217690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91369e2-5fdb-4d15-abf8-4979e30ef67f admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.218192 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91369e2-5fdb-4d15-abf8-4979e30ef67f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:42.220133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91369e2-5fdb-4d15-abf8-4979e30ef67f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:42.221137 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7addc2-5699-47b5-855b-e3793a18c925 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:42.226146 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c696fb-b394-47a7-bcd0-8da22a521764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:42.226772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c696fb-b394-47a7-bcd0-8da22a521764 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.227054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c696fb-b394-47a7-bcd0-8da22a521764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.227317 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c696fb-b394-47a7-bcd0-8da22a521764 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.227983 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c696fb-b394-47a7-bcd0-8da22a521764 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:42.230010 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c696fb-b394-47a7-bcd0-8da22a521764 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:42.257031 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 780/1585] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:33:42 2022] POST /identity/v3/projects => generated 379 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:42.258014 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91369e2-5fdb-4d15-abf8-4979e30ef67f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:42.258285 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 806/1586] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:33:42 2022] POST /identity/v3/users => generated 296 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:42.259133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c696fb-b394-47a7-bcd0-8da22a521764 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:42.272515 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c696fb-b394-47a7-bcd0-8da22a521764 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:42.341293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91369e2-5fdb-4d15-abf8-4979e30ef67f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:42.342932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21995a62-7331-4a79-aa55-b12396e4ed59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:42.343958 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21995a62-7331-4a79-aa55-b12396e4ed59 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.344205 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21995a62-7331-4a79-aa55-b12396e4ed59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.344509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21995a62-7331-4a79-aa55-b12396e4ed59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.345250 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21995a62-7331-4a79-aa55-b12396e4ed59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:42.346468 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720a8bb-7f09-44d3-9932-4bff74b6f4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:42.346917 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21995a62-7331-4a79-aa55-b12396e4ed59 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:42.347105 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720a8bb-7f09-44d3-9932-4bff74b6f4fb admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.347391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720a8bb-7f09-44d3-9932-4bff74b6f4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.347667 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720a8bb-7f09-44d3-9932-4bff74b6f4fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.348515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720a8bb-7f09-44d3-9932-4bff74b6f4fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:42.350096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720a8bb-7f09-44d3-9932-4bff74b6f4fb admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:42.363077 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 807/1587] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:42.363518 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 781/1588] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:42.364396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720a8bb-7f09-44d3-9932-4bff74b6f4fb admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:42.365854 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d55d13-9802-43a9-abfb-91d3ce75f71f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.366191 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d55d13-9802-43a9-abfb-91d3ce75f71f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.366536 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d55d13-9802-43a9-abfb-91d3ce75f71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.380583 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720a8bb-7f09-44d3-9932-4bff74b6f4fb admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:42.387292 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c5d55d13-9802-43a9-abfb-91d3ce75f71f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:42.388532 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c5d55d13-9802-43a9-abfb-91d3ce75f71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:42.389305 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2409903-80ab-434c-9c24-ac4080e6fa8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:42.390170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2409903-80ab-434c-9c24-ac4080e6fa8c admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.390494 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2409903-80ab-434c-9c24-ac4080e6fa8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.390912 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2409903-80ab-434c-9c24-ac4080e6fa8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.391797 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2409903-80ab-434c-9c24-ac4080e6fa8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:42.393754 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2409903-80ab-434c-9c24-ac4080e6fa8c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:42.396466 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 808/1589] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:42.397746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9fa57-acfa-497b-9359-3b872485c79e None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.398061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9fa57-acfa-497b-9359-3b872485c79e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.398332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9fa57-acfa-497b-9359-3b872485c79e None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.399800 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 809/1590] 10.222.0.83 () {54 vars in 962 bytes} [Mon Aug 1 15:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:33:42.404672 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9fa57-acfa-497b-9359-3b872485c79e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:42.414048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 782/1591] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:42.414893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2409903-80ab-434c-9c24-ac4080e6fa8c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:42.419213 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9fa57-acfa-497b-9359-3b872485c79e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:42.424964 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c79e60-91f2-4aef-88ab-d9c570746ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:42.425698 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2409903-80ab-434c-9c24-ac4080e6fa8c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:42.425852 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c79e60-91f2-4aef-88ab-d9c570746ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.426171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c79e60-91f2-4aef-88ab-d9c570746ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.426502 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c79e60-91f2-4aef-88ab-d9c570746ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.427486 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c79e60-91f2-4aef-88ab-d9c570746ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:42.428988 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c79e60-91f2-4aef-88ab-d9c570746ca7 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:42.434016 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f4b0a-dab2-466d-a11e-0fcce5cb3e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:42.434650 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f4b0a-dab2-466d-a11e-0fcce5cb3e79 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.434934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f4b0a-dab2-466d-a11e-0fcce5cb3e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.435207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f4b0a-dab2-466d-a11e-0fcce5cb3e79 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.435880 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f4b0a-dab2-466d-a11e-0fcce5cb3e79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:42.438446 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f4b0a-dab2-466d-a11e-0fcce5cb3e79 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:42.441039 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 810/1592] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:42.441833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c79e60-91f2-4aef-88ab-d9c570746ca7 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:42.454041 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c79e60-91f2-4aef-88ab-d9c570746ca7 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:42.460507 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88101e1-1322-4c9a-9604-574d8ba94e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:42.461148 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88101e1-1322-4c9a-9604-574d8ba94e36 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.461431 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88101e1-1322-4c9a-9604-574d8ba94e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.461743 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88101e1-1322-4c9a-9604-574d8ba94e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.463058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88101e1-1322-4c9a-9604-574d8ba94e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:42.464173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88101e1-1322-4c9a-9604-574d8ba94e36 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:42.468860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 783/1593] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:33:42 2022] POST /identity/v3/users => generated 290 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:42.468860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f4b0a-dab2-466d-a11e-0fcce5cb3e79 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:42.478509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 811/1594] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:42.479889 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f4b0a-dab2-466d-a11e-0fcce5cb3e79 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:42.480211 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88101e1-1322-4c9a-9604-574d8ba94e36 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:42.486928 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491a430-79d6-4aa6-89c7-c9cbc52777c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:42.488335 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491a430-79d6-4aa6-89c7-c9cbc52777c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.488953 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491a430-79d6-4aa6-89c7-c9cbc52777c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.489799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491a430-79d6-4aa6-89c7-c9cbc52777c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.490062 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491a430-79d6-4aa6-89c7-c9cbc52777c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:42.493354 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491a430-79d6-4aa6-89c7-c9cbc52777c8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:42.496406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88101e1-1322-4c9a-9604-574d8ba94e36 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:42.508694 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 784/1595] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:42.508908 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780c12d-1706-4f35-ab9c-6b51519ef239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:42.509975 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780c12d-1706-4f35-ab9c-6b51519ef239 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.510194 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae078f-ac8d-499d-b669-183d93b95461 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.510447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780c12d-1706-4f35-ab9c-6b51519ef239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.510604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae078f-ac8d-499d-b669-183d93b95461 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.510948 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae078f-ac8d-499d-b669-183d93b95461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.511080 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780c12d-1706-4f35-ab9c-6b51519ef239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.512340 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780c12d-1706-4f35-ab9c-6b51519ef239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:42.515328 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780c12d-1706-4f35-ab9c-6b51519ef239 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:42.527637 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 812/1596] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:42.528739 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780c12d-1706-4f35-ab9c-6b51519ef239 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:42.554310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-deae078f-ac8d-499d-b669-183d93b95461 None None] MFA Rules not processed for user `25dc41ffc41c47fdbeec06110d04610c`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:42.555233 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:42.555540 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-deae078f-ac8d-499d-b669-183d93b95461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:42.588988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:42.591780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780c12d-1706-4f35-ab9c-6b51519ef239 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:42.596587 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dfc064-88be-439e-8648-fc3f49d01e0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:42.597224 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfc064-88be-439e-8648-fc3f49d01e0c service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.597508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfc064-88be-439e-8648-fc3f49d01e0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.597966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfc064-88be-439e-8648-fc3f49d01e0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.607781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dfc064-88be-439e-8648-fc3f49d01e0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:42.609287 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dfc064-88be-439e-8648-fc3f49d01e0c service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:42.619597 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 813/1597] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:42.620538 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dfc064-88be-439e-8648-fc3f49d01e0c service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:42.630978 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dfc064-88be-439e-8648-fc3f49d01e0c service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:42.641090 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 785/1598] 10.222.0.83 () {58 vars in 1059 bytes} [Mon Aug 1 15:33:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:42.642231 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae078f-ac8d-499d-b669-183d93b95461 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:42.649854 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75a6ce-fc56-4843-b00d-14a29c3d4b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:42.650442 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75a6ce-fc56-4843-b00d-14a29c3d4b78 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.650725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75a6ce-fc56-4843-b00d-14a29c3d4b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.651003 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75a6ce-fc56-4843-b00d-14a29c3d4b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.651711 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b75a6ce-fc56-4843-b00d-14a29c3d4b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:42.653709 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b75a6ce-fc56-4843-b00d-14a29c3d4b78 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:42.666476 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae078f-ac8d-499d-b669-183d93b95461 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:42.670053 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 814/1599] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:42.671574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75a6ce-fc56-4843-b00d-14a29c3d4b78 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:42.675896 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a56aeb-71dd-42ad-82d5-23340ce3eec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:42.676814 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a56aeb-71dd-42ad-82d5-23340ce3eec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.677110 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a56aeb-71dd-42ad-82d5-23340ce3eec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.677394 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a56aeb-71dd-42ad-82d5-23340ce3eec3 admin admin] PATH_INFO: `/v3/projects/7ded351d7a1b4a9c8bd4901f8becfe5b/users/4001e8ec57b34f57aae1654fad9217f8/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.689382 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75a6ce-fc56-4843-b00d-14a29c3d4b78 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:42.698645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207d485-5fc3-4d9b-bbd6-2bf3d778c0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:42.699343 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207d485-5fc3-4d9b-bbd6-2bf3d778c0a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.699630 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207d485-5fc3-4d9b-bbd6-2bf3d778c0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.699906 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207d485-5fc3-4d9b-bbd6-2bf3d778c0a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.700712 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207d485-5fc3-4d9b-bbd6-2bf3d778c0a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:42.703294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207d485-5fc3-4d9b-bbd6-2bf3d778c0a3 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:42.705184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a56aeb-71dd-42ad-82d5-23340ce3eec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ded351d7a1b4a9c8bd4901f8becfe5b, user_id=4001e8ec57b34f57aae1654fad9217f8, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:42.708825 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a56aeb-71dd-42ad-82d5-23340ce3eec3 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:42.721293 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 815/1600] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:42.722339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207d485-5fc3-4d9b-bbd6-2bf3d778c0a3 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:42.729305 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 786/1601] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:33:42 2022] PUT /identity/v3/projects/7ded351d7a1b4a9c8bd4901f8becfe5b/users/4001e8ec57b34f57aae1654fad9217f8/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:42.730373 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a56aeb-71dd-42ad-82d5-23340ce3eec3 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:42.737010 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207d485-5fc3-4d9b-bbd6-2bf3d778c0a3 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:42.816199 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8dbfbf-531e-4af8-94a1-b25e7da06a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:42.816971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8dbfbf-531e-4af8-94a1-b25e7da06a71 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.817133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8dbfbf-531e-4af8-94a1-b25e7da06a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.817400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8dbfbf-531e-4af8-94a1-b25e7da06a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.818146 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8dbfbf-531e-4af8-94a1-b25e7da06a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:42.820489 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8dbfbf-531e-4af8-94a1-b25e7da06a71 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:42.827609 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a56aeb-71dd-42ad-82d5-23340ce3eec3 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:42.839151 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 816/1602] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:42.839151 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8dbfbf-531e-4af8-94a1-b25e7da06a71 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:42.839987 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8e709-6363-424a-9963-1377fc4f2ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:42.840837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8e709-6363-424a-9963-1377fc4f2ac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.841209 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8e709-6363-424a-9963-1377fc4f2ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.841839 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8e709-6363-424a-9963-1377fc4f2ac7 admin admin] PATH_INFO: `/v3/projects/04503417fb334760a73bded842054784/users/d8d79401b625483b818a017d53b8fe40/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.850209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8dbfbf-531e-4af8-94a1-b25e7da06a71 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:42.859002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844560e-1700-4754-a49b-e3195c259b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:42.860080 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844560e-1700-4754-a49b-e3195c259b48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:42.860371 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844560e-1700-4754-a49b-e3195c259b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:42.860654 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844560e-1700-4754-a49b-e3195c259b48 admin admin] PATH_INFO: `/v3/projects/f60c07cbebc14cb99276113dc186c525/users/9645dc03b0f548288f567d267d014edb/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:42.866925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8e709-6363-424a-9963-1377fc4f2ac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04503417fb334760a73bded842054784, user_id=d8d79401b625483b818a017d53b8fe40, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:42.870085 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8e709-6363-424a-9963-1377fc4f2ac7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:42.885184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844560e-1700-4754-a49b-e3195c259b48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f60c07cbebc14cb99276113dc186c525, user_id=9645dc03b0f548288f567d267d014edb, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:42.888089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844560e-1700-4754-a49b-e3195c259b48 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:42.895656 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 787/1603] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:33:42 2022] PUT /identity/v3/projects/04503417fb334760a73bded842054784/users/d8d79401b625483b818a017d53b8fe40/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:42.896484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8e709-6363-424a-9963-1377fc4f2ac7 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:42.915831 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 817/1604] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:33:42 2022] PUT /identity/v3/projects/f60c07cbebc14cb99276113dc186c525/users/9645dc03b0f548288f567d267d014edb/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:42.918971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844560e-1700-4754-a49b-e3195c259b48 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:43.021126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844560e-1700-4754-a49b-e3195c259b48 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:43.030347 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8e709-6363-424a-9963-1377fc4f2ac7 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:43.040675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731f482-4059-4f18-9bfc-e60d96371056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:43.041381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731f482-4059-4f18-9bfc-e60d96371056 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:43.041659 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731f482-4059-4f18-9bfc-e60d96371056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:43.041971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731f482-4059-4f18-9bfc-e60d96371056 admin admin] PATH_INFO: `/v3/projects/85354737a87146729a06bf22ddb3fefc/users/ba149d1623e849328e399d6dbdca3e0c/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:43.066067 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:43.067518 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731f482-4059-4f18-9bfc-e60d96371056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85354737a87146729a06bf22ddb3fefc, user_id=ba149d1623e849328e399d6dbdca3e0c, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:43.073396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731f482-4059-4f18-9bfc-e60d96371056 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:43.088516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:43.105202 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 818/1605] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:33:42 2022] PUT /identity/v3/projects/85354737a87146729a06bf22ddb3fefc/users/ba149d1623e849328e399d6dbdca3e0c/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:43.106347 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731f482-4059-4f18-9bfc-e60d96371056 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:43.161921 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d1a35-be38-4b4f-9296-dc14fb0a17dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:43.162958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d1a35-be38-4b4f-9296-dc14fb0a17dd service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:43.163313 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d1a35-be38-4b4f-9296-dc14fb0a17dd service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:43.163700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d1a35-be38-4b4f-9296-dc14fb0a17dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:43.211491 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731f482-4059-4f18-9bfc-e60d96371056 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:43.219353 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b273ea1-6d77-4bee-971e-c4fde73b6077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:43.220497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b273ea1-6d77-4bee-971e-c4fde73b6077 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:43.221062 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b273ea1-6d77-4bee-971e-c4fde73b6077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:43.221938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b273ea1-6d77-4bee-971e-c4fde73b6077 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:43.223242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b273ea1-6d77-4bee-971e-c4fde73b6077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:43.229136 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b273ea1-6d77-4bee-971e-c4fde73b6077 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:43.245471 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 819/1606] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:43.246848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffba5f-d2ba-4aa5-8d4b-1538090d76f3 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:43.247141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffba5f-d2ba-4aa5-8d4b-1538090d76f3 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:43.247421 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffba5f-d2ba-4aa5-8d4b-1538090d76f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:43.257340 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d1a35-be38-4b4f-9296-dc14fb0a17dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:43.259101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d1a35-be38-4b4f-9296-dc14fb0a17dd service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:43.269581 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 788/1607] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:43.270412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d1a35-be38-4b4f-9296-dc14fb0a17dd service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:43.272228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-ccffba5f-d2ba-4aa5-8d4b-1538090d76f3 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:43.273574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ccffba5f-d2ba-4aa5-8d4b-1538090d76f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:43.283086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d1a35-be38-4b4f-9296-dc14fb0a17dd service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:43.287364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 820/1608] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:43.287364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fccbf-ddc4-4436-be73-b0e16900c5a8 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:43.287364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fccbf-ddc4-4436-be73-b0e16900c5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:43.296992 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fccbf-ddc4-4436-be73-b0e16900c5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:43.298785 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc0d9e-4c63-4e07-88ff-a2167abe8c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:43.299490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc0d9e-4c63-4e07-88ff-a2167abe8c03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:43.299768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc0d9e-4c63-4e07-88ff-a2167abe8c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:43.300056 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc0d9e-4c63-4e07-88ff-a2167abe8c03 admin admin] PATH_INFO: `/v3/projects/2d49f0b3378c42ada96c61586a38136e/users/5f0b12ba782244d7b44783c3f3532bfa/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:43.318401 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc0d9e-4c63-4e07-88ff-a2167abe8c03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d49f0b3378c42ada96c61586a38136e, user_id=5f0b12ba782244d7b44783c3f3532bfa, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:43.320617 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc0d9e-4c63-4e07-88ff-a2167abe8c03 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:43.336456 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-826fccbf-ddc4-4436-be73-b0e16900c5a8 None None] MFA Rules not processed for user `d8d79401b625483b818a017d53b8fe40`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:43.337460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 789/1609] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:33:43 2022] PUT /identity/v3/projects/2d49f0b3378c42ada96c61586a38136e/users/5f0b12ba782244d7b44783c3f3532bfa/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:43.343823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-826fccbf-ddc4-4436-be73-b0e16900c5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:43.344022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac04860-538a-471c-8206-aa665db3ae17 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:43.344022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac04860-538a-471c-8206-aa665db3ae17 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:43.344022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac04860-538a-471c-8206-aa665db3ae17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:43.386601 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-fac04860-538a-471c-8206-aa665db3ae17 None None] MFA Rules not processed for user `9645dc03b0f548288f567d267d014edb`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:43.388116 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-fac04860-538a-471c-8206-aa665db3ae17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:43.452244 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 821/1610] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:43.454077 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3a099-f100-4ca7-86fb-42e5a7d0cf7d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:43.454326 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3a099-f100-4ca7-86fb-42e5a7d0cf7d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:43.454666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3a099-f100-4ca7-86fb-42e5a7d0cf7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:43.480655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f3e3a099-f100-4ca7-86fb-42e5a7d0cf7d None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:43.483851 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 790/1611] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:43.485054 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac04860-538a-471c-8206-aa665db3ae17 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:43.487597 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f3e3a099-f100-4ca7-86fb-42e5a7d0cf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:43.521080 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:43.571847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac04860-538a-471c-8206-aa665db3ae17 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:43.586594 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefed0b1-d055-4a8b-86f4-f4579646d7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:43.588223 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefed0b1-d055-4a8b-86f4-f4579646d7b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:43.589064 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefed0b1-d055-4a8b-86f4-f4579646d7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:43.589504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefed0b1-d055-4a8b-86f4-f4579646d7b7 admin admin] PATH_INFO: `/v3/projects/db84eff445df4738a5fccb348fe9afeb/users/c07d9c30b18a4150aa0bc7a1705551d7/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:43.595646 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 822/1612] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:43.596762 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3a099-f100-4ca7-86fb-42e5a7d0cf7d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:43.620229 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefed0b1-d055-4a8b-86f4-f4579646d7b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db84eff445df4738a5fccb348fe9afeb, user_id=c07d9c30b18a4150aa0bc7a1705551d7, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:43.623679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:43.628114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefed0b1-d055-4a8b-86f4-f4579646d7b7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:43.661282 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 791/1613] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:33:43 2022] PUT /identity/v3/projects/db84eff445df4738a5fccb348fe9afeb/users/c07d9c30b18a4150aa0bc7a1705551d7/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:43.666066 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac3d5c-034f-4233-b478-d6cbe2821521 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:43.666477 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac3d5c-034f-4233-b478-d6cbe2821521 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:43.667475 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac3d5c-034f-4233-b478-d6cbe2821521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:43.694591 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3a099-f100-4ca7-86fb-42e5a7d0cf7d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:43.698271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-40ac3d5c-034f-4233-b478-d6cbe2821521 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:43.699816 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-40ac3d5c-034f-4233-b478-d6cbe2821521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:43.761909 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d95257-aa52-4a3c-bd29-9771bc0d85bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:43.762799 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d95257-aa52-4a3c-bd29-9771bc0d85bf service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:43.763151 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d95257-aa52-4a3c-bd29-9771bc0d85bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:43.763493 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d95257-aa52-4a3c-bd29-9771bc0d85bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:43.789459 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 792/1614] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:43.790472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac3d5c-034f-4233-b478-d6cbe2821521 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:43.812358 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac3d5c-034f-4233-b478-d6cbe2821521 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:43.826920 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208e1ef-a277-4dc6-a2e0-92efc48e4356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:43.827933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208e1ef-a277-4dc6-a2e0-92efc48e4356 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:43.828371 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208e1ef-a277-4dc6-a2e0-92efc48e4356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:43.828836 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208e1ef-a277-4dc6-a2e0-92efc48e4356 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:43.830038 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a208e1ef-a277-4dc6-a2e0-92efc48e4356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:43.834004 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a208e1ef-a277-4dc6-a2e0-92efc48e4356 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:43.860496 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d95257-aa52-4a3c-bd29-9771bc0d85bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:43.863778 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d95257-aa52-4a3c-bd29-9771bc0d85bf service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:43.876870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 793/1615] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:33:43 2022] POST /identity/v3/projects => generated 381 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:43.878533 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208e1ef-a277-4dc6-a2e0-92efc48e4356 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:43.883899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 823/1616] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:43.885810 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48d1a3-0ecd-4726-bdd3-4c715b636fc0 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:43.886792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48d1a3-0ecd-4726-bdd3-4c715b636fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:43.887086 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48d1a3-0ecd-4726-bdd3-4c715b636fc0 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:43.889378 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 824/1617] 10.222.0.83 () {54 vars in 912 bytes} [Mon Aug 1 15:33:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:33:43.890380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48d1a3-0ecd-4726-bdd3-4c715b636fc0 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:43.982785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48d1a3-0ecd-4726-bdd3-4c715b636fc0 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:43.993119 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0553ea-5587-42fb-884d-4f4bbb3d3d96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:43.993781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0553ea-5587-42fb-884d-4f4bbb3d3d96 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:43.994095 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0553ea-5587-42fb-884d-4f4bbb3d3d96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:43.994260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208e1ef-a277-4dc6-a2e0-92efc48e4356 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:43.994393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0553ea-5587-42fb-884d-4f4bbb3d3d96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:44.000214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dee701-604a-4b53-8939-fa6cbbcc43ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:44.000864 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dee701-604a-4b53-8939-fa6cbbcc43ff admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:44.001232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dee701-604a-4b53-8939-fa6cbbcc43ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:44.001501 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dee701-604a-4b53-8939-fa6cbbcc43ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:44.002238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dee701-604a-4b53-8939-fa6cbbcc43ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:44.004439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dee701-604a-4b53-8939-fa6cbbcc43ff admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:44.007792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0553ea-5587-42fb-884d-4f4bbb3d3d96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:44.009483 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0553ea-5587-42fb-884d-4f4bbb3d3d96 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:44.026207 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 825/1618] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:44.028049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32574b-0889-4291-9c6d-a325f87acf97 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:44.028354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32574b-0889-4291-9c6d-a325f87acf97 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:44.028679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32574b-0889-4291-9c6d-a325f87acf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:44.049228 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 794/1619] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:33:43 2022] POST /identity/v3/projects => generated 405 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:44.049228 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dee701-604a-4b53-8939-fa6cbbcc43ff admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:44.082163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:44.089592 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-fb32574b-0889-4291-9c6d-a325f87acf97 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:44.093428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fb32574b-0889-4291-9c6d-a325f87acf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:44.148252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dee701-604a-4b53-8939-fa6cbbcc43ff admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:44.149384 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:44.154755 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c51ce-e3f8-4939-be14-9fa5f2b2446e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:44.156279 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c51ce-e3f8-4939-be14-9fa5f2b2446e service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:44.156744 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c51ce-e3f8-4939-be14-9fa5f2b2446e service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:44.157174 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c51ce-e3f8-4939-be14-9fa5f2b2446e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:44.246138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c51ce-e3f8-4939-be14-9fa5f2b2446e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:44.247407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c51ce-e3f8-4939-be14-9fa5f2b2446e service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:44.257865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 826/1620] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:44.258894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32574b-0889-4291-9c6d-a325f87acf97 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:44.269806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 795/1621] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:44.272986 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6376378-8ac2-42fd-bbdd-c5c05348239f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:44.273089 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6376378-8ac2-42fd-bbdd-c5c05348239f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:44.273376 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6376378-8ac2-42fd-bbdd-c5c05348239f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:44.287083 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32574b-0889-4291-9c6d-a325f87acf97 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:44.303355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f6376378-8ac2-42fd-bbdd-c5c05348239f None None] MFA Rules not processed for user `25dc41ffc41c47fdbeec06110d04610c`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:44.304760 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f6376378-8ac2-42fd-bbdd-c5c05348239f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:44.307744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0564c33-f317-4aaf-920d-a7136512fbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:44.307744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0564c33-f317-4aaf-920d-a7136512fbb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:44.307744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0564c33-f317-4aaf-920d-a7136512fbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:44.307744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0564c33-f317-4aaf-920d-a7136512fbb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:44.308264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0564c33-f317-4aaf-920d-a7136512fbb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:44.310252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0564c33-f317-4aaf-920d-a7136512fbb5 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:44.378008 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 827/1622] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:33:44 2022] POST /identity/v3/users => generated 297 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:44.378953 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0564c33-f317-4aaf-920d-a7136512fbb5 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:44.402206 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0564c33-f317-4aaf-920d-a7136512fbb5 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:44.414918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78bfb5-57b0-4fc1-b4e4-fe7e44d31f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:44.415643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78bfb5-57b0-4fc1-b4e4-fe7e44d31f0b admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:44.415850 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78bfb5-57b0-4fc1-b4e4-fe7e44d31f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:44.416131 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78bfb5-57b0-4fc1-b4e4-fe7e44d31f0b admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:44.416813 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78bfb5-57b0-4fc1-b4e4-fe7e44d31f0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:44.423589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78bfb5-57b0-4fc1-b4e4-fe7e44d31f0b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:44.437153 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 796/1623] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Aug 1 15:33:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:44.438249 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6376378-8ac2-42fd-bbdd-c5c05348239f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:44.450757 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6376378-8ac2-42fd-bbdd-c5c05348239f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:44.457411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eee3e5-f90e-4d65-a23f-002855aa2f72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:44.458552 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eee3e5-f90e-4d65-a23f-002855aa2f72 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:44.459002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eee3e5-f90e-4d65-a23f-002855aa2f72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:44.459381 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eee3e5-f90e-4d65-a23f-002855aa2f72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:44.471719 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 828/1624] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:33:44 2022] POST /identity/v3/users => generated 309 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:44.472721 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78bfb5-57b0-4fc1-b4e4-fe7e44d31f0b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:44.476517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eee3e5-f90e-4d65-a23f-002855aa2f72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:44.479478 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eee3e5-f90e-4d65-a23f-002855aa2f72 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:44.494819 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78bfb5-57b0-4fc1-b4e4-fe7e44d31f0b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:44.495612 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 797/1625] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:44.496555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eee3e5-f90e-4d65-a23f-002855aa2f72 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:44.510482 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392a41a-20a9-49c1-bd66-e7cea7d731f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:44.511258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392a41a-20a9-49c1-bd66-e7cea7d731f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:44.511575 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392a41a-20a9-49c1-bd66-e7cea7d731f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:44.511855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392a41a-20a9-49c1-bd66-e7cea7d731f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:44.512637 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7392a41a-20a9-49c1-bd66-e7cea7d731f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:44.516130 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7392a41a-20a9-49c1-bd66-e7cea7d731f4 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:44.548033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 829/1626] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:44.549062 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392a41a-20a9-49c1-bd66-e7cea7d731f4 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:44.568603 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:44.568835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392a41a-20a9-49c1-bd66-e7cea7d731f4 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:44.583772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062ff92-a20a-4ffb-b7ce-f6cb236fe91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:44.584508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062ff92-a20a-4ffb-b7ce-f6cb236fe91f admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:44.584698 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062ff92-a20a-4ffb-b7ce-f6cb236fe91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:44.584979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062ff92-a20a-4ffb-b7ce-f6cb236fe91f admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:44.586408 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062ff92-a20a-4ffb-b7ce-f6cb236fe91f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:44.591746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062ff92-a20a-4ffb-b7ce-f6cb236fe91f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:44.598185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eee3e5-f90e-4d65-a23f-002855aa2f72 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:44.603592 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb1243-96ce-46a0-9283-2049f4d0284b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:44.604706 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb1243-96ce-46a0-9283-2049f4d0284b service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:44.604914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb1243-96ce-46a0-9283-2049f4d0284b service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:44.605238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb1243-96ce-46a0-9283-2049f4d0284b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:44.622561 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb1243-96ce-46a0-9283-2049f4d0284b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:44.622797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 830/1627] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:44.623532 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062ff92-a20a-4ffb-b7ce-f6cb236fe91f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:44.624736 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb1243-96ce-46a0-9283-2049f4d0284b service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:44.657427 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 798/1628] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:44.662131 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062ff92-a20a-4ffb-b7ce-f6cb236fe91f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:44.673795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb1243-96ce-46a0-9283-2049f4d0284b service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:44.676359 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca0f2a-7044-432c-8e8e-8e8a9e24c02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:44.677419 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca0f2a-7044-432c-8e8e-8e8a9e24c02c admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:44.682114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca0f2a-7044-432c-8e8e-8e8a9e24c02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:44.682584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca0f2a-7044-432c-8e8e-8e8a9e24c02c admin admin] PATH_INFO: `/v3/projects/f092da46cb5748439ca4c0ee8dda6489/users/3cad08ce5bc04bc0ad6a1feb606f2935/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:44.706655 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb1243-96ce-46a0-9283-2049f4d0284b service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:44.720388 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a709c-51ad-4f3f-8702-0589fb5a40b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:44.721073 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a709c-51ad-4f3f-8702-0589fb5a40b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:44.721409 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a709c-51ad-4f3f-8702-0589fb5a40b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:44.721835 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a709c-51ad-4f3f-8702-0589fb5a40b5 admin admin] PATH_INFO: `/v3/projects/67f382b66b50423cb7bec1824a2cd9a8/users/10c7a43493fb4a008d38d4be479d0cb0/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:44.741407 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca0f2a-7044-432c-8e8e-8e8a9e24c02c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f092da46cb5748439ca4c0ee8dda6489, user_id=3cad08ce5bc04bc0ad6a1feb606f2935, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:44.750023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca0f2a-7044-432c-8e8e-8e8a9e24c02c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:44.763081 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a709c-51ad-4f3f-8702-0589fb5a40b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67f382b66b50423cb7bec1824a2cd9a8, user_id=10c7a43493fb4a008d38d4be479d0cb0, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:44.766741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a709c-51ad-4f3f-8702-0589fb5a40b5 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:44.795282 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 799/1629] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:33:44 2022] PUT /identity/v3/projects/67f382b66b50423cb7bec1824a2cd9a8/users/10c7a43493fb4a008d38d4be479d0cb0/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:44.798559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 831/1630] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:33:44 2022] PUT /identity/v3/projects/f092da46cb5748439ca4c0ee8dda6489/users/3cad08ce5bc04bc0ad6a1feb606f2935/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:44.799862 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a709c-51ad-4f3f-8702-0589fb5a40b5 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:44.805085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca0f2a-7044-432c-8e8e-8e8a9e24c02c admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:44.955685 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a709c-51ad-4f3f-8702-0589fb5a40b5 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:44.973389 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca0f2a-7044-432c-8e8e-8e8a9e24c02c admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:44.976239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f66ab-64a3-4abd-b202-c5a489cd3a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:44.977396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f66ab-64a3-4abd-b202-c5a489cd3a96 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:44.977917 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f66ab-64a3-4abd-b202-c5a489cd3a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:44.982457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f66ab-64a3-4abd-b202-c5a489cd3a96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:44.983390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218af26-5a12-467e-8985-3aa66749254b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:44.983752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f66ab-64a3-4abd-b202-c5a489cd3a96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:44.984106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218af26-5a12-467e-8985-3aa66749254b admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:44.984392 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218af26-5a12-467e-8985-3aa66749254b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:44.984672 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218af26-5a12-467e-8985-3aa66749254b admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:44.985390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4218af26-5a12-467e-8985-3aa66749254b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:44.985517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f66ab-64a3-4abd-b202-c5a489cd3a96 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:44.991122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4218af26-5a12-467e-8985-3aa66749254b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:45.005635 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 800/1631] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:45.007391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f66ab-64a3-4abd-b202-c5a489cd3a96 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:45.105402 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 832/1632] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:33:45.105402 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218af26-5a12-467e-8985-3aa66749254b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:45.106370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f66ab-64a3-4abd-b202-c5a489cd3a96 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:45.106370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096b4b5-782e-41a3-939c-b755ed6ad237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:45.106370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096b4b5-782e-41a3-939c-b755ed6ad237 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:45.106370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096b4b5-782e-41a3-939c-b755ed6ad237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:45.107067 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096b4b5-782e-41a3-939c-b755ed6ad237 admin admin] PATH_INFO: `/v3/projects/67f382b66b50423cb7bec1824a2cd9a8/users/10c7a43493fb4a008d38d4be479d0cb0/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:45.107067 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096b4b5-782e-41a3-939c-b755ed6ad237 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67f382b66b50423cb7bec1824a2cd9a8, user_id=10c7a43493fb4a008d38d4be479d0cb0, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:45.107067 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096b4b5-782e-41a3-939c-b755ed6ad237 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:45.107344 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218af26-5a12-467e-8985-3aa66749254b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:45.107344 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515aa25-faad-4481-9fb8-0a9e620f4a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:45.107344 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515aa25-faad-4481-9fb8-0a9e620f4a0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:45.107344 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515aa25-faad-4481-9fb8-0a9e620f4a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:45.107717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515aa25-faad-4481-9fb8-0a9e620f4a0d admin admin] PATH_INFO: `/v3/projects/f092da46cb5748439ca4c0ee8dda6489/users/3cad08ce5bc04bc0ad6a1feb606f2935/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:45.107817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 801/1633] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:33:45 2022] PUT /identity/v3/projects/67f382b66b50423cb7bec1824a2cd9a8/users/10c7a43493fb4a008d38d4be479d0cb0/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:33:45.107817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1eea30-a34e-44f7-a380-f4cd502f6371 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:45.107997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515aa25-faad-4481-9fb8-0a9e620f4a0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f092da46cb5748439ca4c0ee8dda6489, user_id=3cad08ce5bc04bc0ad6a1feb606f2935, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:45.108160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1eea30-a34e-44f7-a380-f4cd502f6371 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:45.108160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1eea30-a34e-44f7-a380-f4cd502f6371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:45.109136 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515aa25-faad-4481-9fb8-0a9e620f4a0d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:45.141620 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 833/1634] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:33:45 2022] PUT /identity/v3/projects/f092da46cb5748439ca4c0ee8dda6489/users/3cad08ce5bc04bc0ad6a1feb606f2935/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:45.152220 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede5f90-dd7e-4e7b-bc19-f06865febe24 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:45.152506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede5f90-dd7e-4e7b-bc19-f06865febe24 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:45.152779 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede5f90-dd7e-4e7b-bc19-f06865febe24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:45.189419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-1a1eea30-a34e-44f7-a380-f4cd502f6371 None None] MFA Rules not processed for user `10c7a43493fb4a008d38d4be479d0cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:45.192518 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1a1eea30-a34e-44f7-a380-f4cd502f6371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:45.195337 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8ede5f90-dd7e-4e7b-bc19-f06865febe24 None None] MFA Rules not processed for user `3cad08ce5bc04bc0ad6a1feb606f2935`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:45.196682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8ede5f90-dd7e-4e7b-bc19-f06865febe24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:45.225221 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:45.335886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 834/1635] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:45.347176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede5f90-dd7e-4e7b-bc19-f06865febe24 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:45.393332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 802/1636] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:45 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:45.464601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede5f90-dd7e-4e7b-bc19-f06865febe24 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:45.474081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99c042-c72a-4228-bb18-a27cd044fe4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:45.474932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99c042-c72a-4228-bb18-a27cd044fe4a service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:45.475297 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99c042-c72a-4228-bb18-a27cd044fe4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:45.475650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99c042-c72a-4228-bb18-a27cd044fe4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:45.502909 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99c042-c72a-4228-bb18-a27cd044fe4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:45.505607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99c042-c72a-4228-bb18-a27cd044fe4a service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:45.523357 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 835/1637] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:33:45.602617 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1eea30-a34e-44f7-a380-f4cd502f6371 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:45.615950 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1eea30-a34e-44f7-a380-f4cd502f6371 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:45.622470 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75601e65-faa1-484d-a015-6fb4e2208318 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:45.624204 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75601e65-faa1-484d-a015-6fb4e2208318 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:45.624713 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75601e65-faa1-484d-a015-6fb4e2208318 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:45.625258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75601e65-faa1-484d-a015-6fb4e2208318 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:45.688385 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75601e65-faa1-484d-a015-6fb4e2208318 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:45.698008 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75601e65-faa1-484d-a015-6fb4e2208318 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:45.770418 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 803/1638] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:45 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:45.987386 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99c042-c72a-4228-bb18-a27cd044fe4a service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:46.005460 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99c042-c72a-4228-bb18-a27cd044fe4a service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:46.015101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7dd26c-c9a2-438f-95ea-f74e6500794d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:46.021626 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7dd26c-c9a2-438f-95ea-f74e6500794d service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:46.022858 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7dd26c-c9a2-438f-95ea-f74e6500794d service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:46.023419 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7dd26c-c9a2-438f-95ea-f74e6500794d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:46.349998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7dd26c-c9a2-438f-95ea-f74e6500794d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:46.360031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7dd26c-c9a2-438f-95ea-f74e6500794d service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:46.416074 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 836/1639] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:46.531458 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75601e65-faa1-484d-a015-6fb4e2208318 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:46.573472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:46.805170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75601e65-faa1-484d-a015-6fb4e2208318 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:46.820330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b01c99-1fa4-4467-b2cc-d60047c4402a None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:46.820330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b01c99-1fa4-4467-b2cc-d60047c4402a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:46.821306 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b01c99-1fa4-4467-b2cc-d60047c4402a None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:46.827818 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 837/1640] 10.222.0.83 () {54 vars in 962 bytes} [Mon Aug 1 15:33:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:33:46.843252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07eb5f-a99b-40b5-a9e4-187c87838597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:46.849155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07eb5f-a99b-40b5-a9e4-187c87838597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:46.849661 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07eb5f-a99b-40b5-a9e4-187c87838597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:46.850085 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07eb5f-a99b-40b5-a9e4-187c87838597 admin admin] PATH_INFO: `/v3/users/9645dc03b0f548288f567d267d014edb` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:46.853956 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cfaad-7ad8-438c-978e-96dd95af178a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:46.854432 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cfaad-7ad8-438c-978e-96dd95af178a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:46.854806 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cfaad-7ad8-438c-978e-96dd95af178a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:46.865208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad07eb5f-a99b-40b5-a9e4-187c87838597 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9645dc03b0f548288f567d267d014edb)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:46.872230 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: WARNING py.warnings [None req-ad07eb5f-a99b-40b5-a9e4-187c87838597 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:33:46.872230 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: warnings.warn(msg) Aug 01 15:33:46.872230 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: Aug 01 15:33:46.886396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad07eb5f-a99b-40b5-a9e4-187c87838597 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:46.917020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-aa9cfaad-7ad8-438c-978e-96dd95af178a None None] MFA Rules not processed for user `25dc41ffc41c47fdbeec06110d04610c`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:46.919004 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-aa9cfaad-7ad8-438c-978e-96dd95af178a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:47.059654 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:47.100739 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ad07eb5f-a99b-40b5-a9e4-187c87838597 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9645dc03b0f548288f567d267d014edb'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:47.102155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ad07eb5f-a99b-40b5-a9e4-187c87838597 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9645dc03b0f548288f567d267d014edb'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:47.116906 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ad07eb5f-a99b-40b5-a9e4-187c87838597 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9645dc03b0f548288f567d267d014edb'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:47.147613 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ad07eb5f-a99b-40b5-a9e4-187c87838597 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9645dc03b0f548288f567d267d014edb'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:47.148697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 838/1641] 10.222.0.83 () {58 vars in 1059 bytes} [Mon Aug 1 15:33:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:33:47.148997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ad07eb5f-a99b-40b5-a9e4-187c87838597 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9645dc03b0f548288f567d267d014edb'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:47.161613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bcb10-50ee-42a9-aa95-c0093318f2a0 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:47.162798 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bcb10-50ee-42a9-aa95-c0093318f2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:47.162798 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bcb10-50ee-42a9-aa95-c0093318f2a0 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:47.163785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 839/1642] 10.222.0.83 () {54 vars in 912 bytes} [Mon Aug 1 15:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:33:47.175052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46f3db-a981-4cba-9d88-1dad88763b64 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:47.175590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46f3db-a981-4cba-9d88-1dad88763b64 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:47.175590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46f3db-a981-4cba-9d88-1dad88763b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:47.184661 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 804/1643] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:33:46 2022] DELETE /identity/v3/users/9645dc03b0f548288f567d267d014edb => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:33:47.192762 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e961f6-dafb-4f95-ba6a-61110a14449d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:47.193443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e961f6-dafb-4f95-ba6a-61110a14449d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:47.194172 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e961f6-dafb-4f95-ba6a-61110a14449d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:47.216702 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1e46f3db-a981-4cba-9d88-1dad88763b64 None None] MFA Rules not processed for user `25dc41ffc41c47fdbeec06110d04610c`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:47.219049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1e46f3db-a981-4cba-9d88-1dad88763b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:47.269633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-89e961f6-dafb-4f95-ba6a-61110a14449d None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:47.275748 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-89e961f6-dafb-4f95-ba6a-61110a14449d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:47.395107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 805/1644] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:47.398425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 840/1645] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Aug 1 15:33:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:33:47.411063 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e961f6-dafb-4f95-ba6a-61110a14449d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:47.412877 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-89e961f6-dafb-4f95-ba6a-61110a14449d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:47.492810 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:47.513462 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46f3db-a981-4cba-9d88-1dad88763b64 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:47.516338 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1e46f3db-a981-4cba-9d88-1dad88763b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:47.555088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e961f6-dafb-4f95-ba6a-61110a14449d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:47.560285 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b2f42-d3a7-4962-ae85-ba7a6bfa8b83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:47.561721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b2f42-d3a7-4962-ae85-ba7a6bfa8b83 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:47.562310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b2f42-d3a7-4962-ae85-ba7a6bfa8b83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:47.563206 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b2f42-d3a7-4962-ae85-ba7a6bfa8b83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:47.585836 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b2f42-d3a7-4962-ae85-ba7a6bfa8b83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:47.587868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b2f42-d3a7-4962-ae85-ba7a6bfa8b83 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:47.608385 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 806/1646] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:47.656242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46f3db-a981-4cba-9d88-1dad88763b64 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:47.668001 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ba0a7-bd8e-4ce5-a001-a8c7d1b90b40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:47.668799 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ba0a7-bd8e-4ce5-a001-a8c7d1b90b40 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:47.669410 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ba0a7-bd8e-4ce5-a001-a8c7d1b90b40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:47.669410 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ba0a7-bd8e-4ce5-a001-a8c7d1b90b40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:47.686678 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ba0a7-bd8e-4ce5-a001-a8c7d1b90b40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:47.689212 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ba0a7-bd8e-4ce5-a001-a8c7d1b90b40 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:47.716366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 841/1647] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:49.077843 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b2f42-d3a7-4962-ae85-ba7a6bfa8b83 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:49.080849 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-125b2f42-d3a7-4962-ae85-ba7a6bfa8b83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:49.122861 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b2f42-d3a7-4962-ae85-ba7a6bfa8b83 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:49.146874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91970c86-8998-4c83-bce6-a34a1cd16790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:49.147567 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91970c86-8998-4c83-bce6-a34a1cd16790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:49.147856 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91970c86-8998-4c83-bce6-a34a1cd16790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:49.148139 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91970c86-8998-4c83-bce6-a34a1cd16790 admin admin] PATH_INFO: `/v3/projects/f60c07cbebc14cb99276113dc186c525` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:49.163346 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91970c86-8998-4c83-bce6-a34a1cd16790 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f60c07cbebc14cb99276113dc186c525)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:49.165479 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91970c86-8998-4c83-bce6-a34a1cd16790 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:49.302287 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-91970c86-8998-4c83-bce6-a34a1cd16790 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f60c07cbebc14cb99276113dc186c525', 'request_id': 'req-91970c86-8998-4c83-bce6-a34a1cd16790'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:49.309471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-91970c86-8998-4c83-bce6-a34a1cd16790 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f60c07cbebc14cb99276113dc186c525', 'request_id': 'req-91970c86-8998-4c83-bce6-a34a1cd16790'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:49.318269 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-91970c86-8998-4c83-bce6-a34a1cd16790 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f60c07cbebc14cb99276113dc186c525', 'request_id': 'req-91970c86-8998-4c83-bce6-a34a1cd16790'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:49.367598 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-91970c86-8998-4c83-bce6-a34a1cd16790 admin admin] The token cache is being invalidate because project f60c07cbebc14cb99276113dc186c525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:33:49.368852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-91970c86-8998-4c83-bce6-a34a1cd16790 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:49.370747 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-91970c86-8998-4c83-bce6-a34a1cd16790 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:49.379773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ba0a7-bd8e-4ce5-a001-a8c7d1b90b40 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:49.381426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 807/1648] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:33:49 2022] DELETE /identity/v3/projects/f60c07cbebc14cb99276113dc186c525 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:49.395063 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ba0a7-bd8e-4ce5-a001-a8c7d1b90b40 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:49.580389 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c67f5-025f-472c-9a78-406398c14e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:49.581068 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c67f5-025f-472c-9a78-406398c14e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:49.581347 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c67f5-025f-472c-9a78-406398c14e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:49.581619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c67f5-025f-472c-9a78-406398c14e51 admin admin] PATH_INFO: `/v3/users/10c7a43493fb4a008d38d4be479d0cb0` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:49.584885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c67f5-025f-472c-9a78-406398c14e51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10c7a43493fb4a008d38d4be479d0cb0)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:49.597514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c67f5-025f-472c-9a78-406398c14e51 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:49.662215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:49.754924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-598c67f5-025f-472c-9a78-406398c14e51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10c7a43493fb4a008d38d4be479d0cb0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:49.756838 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-598c67f5-025f-472c-9a78-406398c14e51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10c7a43493fb4a008d38d4be479d0cb0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:49.772423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-598c67f5-025f-472c-9a78-406398c14e51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10c7a43493fb4a008d38d4be479d0cb0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:49.832774 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-598c67f5-025f-472c-9a78-406398c14e51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10c7a43493fb4a008d38d4be479d0cb0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:49.856893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-598c67f5-025f-472c-9a78-406398c14e51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10c7a43493fb4a008d38d4be479d0cb0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:49.885861 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 842/1649] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:33:49 2022] DELETE /identity/v3/users/10c7a43493fb4a008d38d4be479d0cb0 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:33:51.770532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91970c86-8998-4c83-bce6-a34a1cd16790 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:51.772378 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-91970c86-8998-4c83-bce6-a34a1cd16790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:51.847747 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91970c86-8998-4c83-bce6-a34a1cd16790 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:52.044566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba60016-48d7-4115-b71a-da8b56ebf065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:52.059414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba60016-48d7-4115-b71a-da8b56ebf065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:52.070258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba60016-48d7-4115-b71a-da8b56ebf065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:52.070705 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba60016-48d7-4115-b71a-da8b56ebf065 admin admin] PATH_INFO: `/v3/projects/67f382b66b50423cb7bec1824a2cd9a8` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:52.072918 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba60016-48d7-4115-b71a-da8b56ebf065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67f382b66b50423cb7bec1824a2cd9a8)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:52.078419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba60016-48d7-4115-b71a-da8b56ebf065 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:52.186163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-eba60016-48d7-4115-b71a-da8b56ebf065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67f382b66b50423cb7bec1824a2cd9a8', 'request_id': 'req-eba60016-48d7-4115-b71a-da8b56ebf065'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:52.201044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-eba60016-48d7-4115-b71a-da8b56ebf065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67f382b66b50423cb7bec1824a2cd9a8', 'request_id': 'req-eba60016-48d7-4115-b71a-da8b56ebf065'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:52.223393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:52.226175 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-eba60016-48d7-4115-b71a-da8b56ebf065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67f382b66b50423cb7bec1824a2cd9a8', 'request_id': 'req-eba60016-48d7-4115-b71a-da8b56ebf065'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:52.255217 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-eba60016-48d7-4115-b71a-da8b56ebf065 admin admin] The token cache is being invalidate because project 67f382b66b50423cb7bec1824a2cd9a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:33:52.255449 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-eba60016-48d7-4115-b71a-da8b56ebf065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:52.259447 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-eba60016-48d7-4115-b71a-da8b56ebf065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:52.262100 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 808/1650] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:33:51 2022] DELETE /identity/v3/projects/67f382b66b50423cb7bec1824a2cd9a8 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:52.267588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797e3d3-7fee-41cb-a051-e47cf3db0c22 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:52.267995 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797e3d3-7fee-41cb-a051-e47cf3db0c22 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:52.268697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797e3d3-7fee-41cb-a051-e47cf3db0c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:52.307431 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e797e3d3-7fee-41cb-a051-e47cf3db0c22 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:52.314435 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e797e3d3-7fee-41cb-a051-e47cf3db0c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:52.504558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 843/1651] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:52.513765 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4a32f-16be-4072-8c17-1fd66e313ece None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:52.514103 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4a32f-16be-4072-8c17-1fd66e313ece None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:52.515358 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4a32f-16be-4072-8c17-1fd66e313ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:52.567247 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-01d4a32f-16be-4072-8c17-1fd66e313ece None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:52.568909 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-01d4a32f-16be-4072-8c17-1fd66e313ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:52.587239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 809/1652] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:52.591554 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797e3d3-7fee-41cb-a051-e47cf3db0c22 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:52.619538 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797e3d3-7fee-41cb-a051-e47cf3db0c22 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:52.626872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b09553-8be5-4c20-bec2-05ba38669949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:52.627726 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b09553-8be5-4c20-bec2-05ba38669949 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:52.628667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b09553-8be5-4c20-bec2-05ba38669949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:52.629142 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b09553-8be5-4c20-bec2-05ba38669949 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:52.630085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b09553-8be5-4c20-bec2-05ba38669949 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:33:52.630505 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b09553-8be5-4c20-bec2-05ba38669949 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:52.632890 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b09553-8be5-4c20-bec2-05ba38669949 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:52.663501 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 844/1653] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:33:52.668214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4a32f-16be-4072-8c17-1fd66e313ece None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:52.692133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4a32f-16be-4072-8c17-1fd66e313ece None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:52.701324 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cfa0fd-f2d5-4e4f-8b92-f8c991335307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:52.703264 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cfa0fd-f2d5-4e4f-8b92-f8c991335307 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:52.703956 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cfa0fd-f2d5-4e4f-8b92-f8c991335307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:52.704470 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cfa0fd-f2d5-4e4f-8b92-f8c991335307 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:52.710951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cfa0fd-f2d5-4e4f-8b92-f8c991335307 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:52.715292 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cfa0fd-f2d5-4e4f-8b92-f8c991335307 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:52.778475 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 810/1654] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:33:52 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:52.785320 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b09553-8be5-4c20-bec2-05ba38669949 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:52.813920 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b09553-8be5-4c20-bec2-05ba38669949 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:53.091053 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da050b9a-f97b-44fb-b92a-ee16ca5f9d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:53.092854 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da050b9a-f97b-44fb-b92a-ee16ca5f9d0e admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:53.093469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da050b9a-f97b-44fb-b92a-ee16ca5f9d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:53.094891 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da050b9a-f97b-44fb-b92a-ee16ca5f9d0e admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:53.096797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da050b9a-f97b-44fb-b92a-ee16ca5f9d0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:53.106636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da050b9a-f97b-44fb-b92a-ee16ca5f9d0e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:53.186655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 845/1655] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:33:52 2022] POST /identity/v3/users => generated 293 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:53.191448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cfa0fd-f2d5-4e4f-8b92-f8c991335307 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:53.221703 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cfa0fd-f2d5-4e4f-8b92-f8c991335307 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:53.238820 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575c75b-9899-4642-a2e2-71afe007b3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:53.240092 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575c75b-9899-4642-a2e2-71afe007b3c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:53.240534 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575c75b-9899-4642-a2e2-71afe007b3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:53.241082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575c75b-9899-4642-a2e2-71afe007b3c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:53.243381 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575c75b-9899-4642-a2e2-71afe007b3c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:53.246682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575c75b-9899-4642-a2e2-71afe007b3c4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:53.276737 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 811/1656] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:53.281868 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da050b9a-f97b-44fb-b92a-ee16ca5f9d0e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:53.327077 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da050b9a-f97b-44fb-b92a-ee16ca5f9d0e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:53.346872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614ca49-5ab0-4144-8237-9c80bfbdb7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:53.348795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614ca49-5ab0-4144-8237-9c80bfbdb7cb admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:53.349626 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614ca49-5ab0-4144-8237-9c80bfbdb7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:53.350309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614ca49-5ab0-4144-8237-9c80bfbdb7cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:53.351714 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614ca49-5ab0-4144-8237-9c80bfbdb7cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:53.362067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614ca49-5ab0-4144-8237-9c80bfbdb7cb admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:53.420196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 846/1657] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:53.430136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575c75b-9899-4642-a2e2-71afe007b3c4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:53.449221 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575c75b-9899-4642-a2e2-71afe007b3c4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:53.461381 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083ec1d-31d2-48fc-a4e9-58f39e397829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:53.462220 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083ec1d-31d2-48fc-a4e9-58f39e397829 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:53.462530 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083ec1d-31d2-48fc-a4e9-58f39e397829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:53.462815 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083ec1d-31d2-48fc-a4e9-58f39e397829 admin admin] PATH_INFO: `/v3/projects/1d47773f079843f3a9d7d41b85a3b739/users/0faa07531d7f4360bfea8673650b0083/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:53.513755 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083ec1d-31d2-48fc-a4e9-58f39e397829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d47773f079843f3a9d7d41b85a3b739, user_id=0faa07531d7f4360bfea8673650b0083, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:53.517240 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083ec1d-31d2-48fc-a4e9-58f39e397829 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:53.553798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 812/1658] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:33:53 2022] PUT /identity/v3/projects/1d47773f079843f3a9d7d41b85a3b739/users/0faa07531d7f4360bfea8673650b0083/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:53.558152 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa1135-1ab5-4b63-b53c-e29dd00905bc None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:53.558562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa1135-1ab5-4b63-b53c-e29dd00905bc None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:53.558716 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa1135-1ab5-4b63-b53c-e29dd00905bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:53.599771 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:53.649958 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-68fa1135-1ab5-4b63-b53c-e29dd00905bc None None] MFA Rules not processed for user `0faa07531d7f4360bfea8673650b0083`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:53.651210 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-68fa1135-1ab5-4b63-b53c-e29dd00905bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:53.852239 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 847/1659] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:53.857068 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083ec1d-31d2-48fc-a4e9-58f39e397829 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:54.047059 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083ec1d-31d2-48fc-a4e9-58f39e397829 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:54.069145 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da113217-0dd3-4341-8e5d-0733a8f2cdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:54.069145 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da113217-0dd3-4341-8e5d-0733a8f2cdad admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:54.070980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da113217-0dd3-4341-8e5d-0733a8f2cdad admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:54.071285 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da113217-0dd3-4341-8e5d-0733a8f2cdad admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:54.076019 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da113217-0dd3-4341-8e5d-0733a8f2cdad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:54.079121 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da113217-0dd3-4341-8e5d-0733a8f2cdad admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:54.167175 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 813/1660] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:33:53 2022] POST /identity/v3/projects => generated 387 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:54.170304 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa1135-1ab5-4b63-b53c-e29dd00905bc None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:54.194682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa1135-1ab5-4b63-b53c-e29dd00905bc None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:54.361377 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b46fd2-a842-4c27-9eea-394ac07d57cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:54.367043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b46fd2-a842-4c27-9eea-394ac07d57cc admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:54.367455 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b46fd2-a842-4c27-9eea-394ac07d57cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:54.367777 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b46fd2-a842-4c27-9eea-394ac07d57cc admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:54.368552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b46fd2-a842-4c27-9eea-394ac07d57cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:54.371124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b46fd2-a842-4c27-9eea-394ac07d57cc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:54.418101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 848/1661] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:33:54 2022] POST /identity/v3/users => generated 300 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:54.418847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da113217-0dd3-4341-8e5d-0733a8f2cdad admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:54.447609 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da113217-0dd3-4341-8e5d-0733a8f2cdad admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:54.457378 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a3f37-6c10-4ed3-b580-018edfb2d78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:54.460132 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a3f37-6c10-4ed3-b580-018edfb2d78f admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:54.460660 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a3f37-6c10-4ed3-b580-018edfb2d78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:54.461117 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a3f37-6c10-4ed3-b580-018edfb2d78f admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:54.462296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a3f37-6c10-4ed3-b580-018edfb2d78f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:54.464704 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a3f37-6c10-4ed3-b580-018edfb2d78f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:54.492697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b46fd2-a842-4c27-9eea-394ac07d57cc admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:54.506629 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 814/1662] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:54.508739 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b46fd2-a842-4c27-9eea-394ac07d57cc admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:54.517150 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2e6a7-1262-4e5e-804b-b6c94e259825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:54.518442 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2e6a7-1262-4e5e-804b-b6c94e259825 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:54.519104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2e6a7-1262-4e5e-804b-b6c94e259825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:54.519659 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2e6a7-1262-4e5e-804b-b6c94e259825 admin admin] PATH_INFO: `/v3/projects/9609419b77ee4a388ea83b34408d4591/users/185575822d8e4f27a9e949b9f8af7d9a/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:54.543011 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2e6a7-1262-4e5e-804b-b6c94e259825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9609419b77ee4a388ea83b34408d4591, user_id=185575822d8e4f27a9e949b9f8af7d9a, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:54.547245 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2e6a7-1262-4e5e-804b-b6c94e259825 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:54.601929 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a3f37-6c10-4ed3-b580-018edfb2d78f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:54.604736 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 849/1663] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:33:54 2022] PUT /identity/v3/projects/9609419b77ee4a388ea83b34408d4591/users/185575822d8e4f27a9e949b9f8af7d9a/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:54.619577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a3f37-6c10-4ed3-b580-018edfb2d78f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:54.752791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e19140-14f6-41f4-a986-3fb5e945b36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:54.755385 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e19140-14f6-41f4-a986-3fb5e945b36c admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:54.755799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e19140-14f6-41f4-a986-3fb5e945b36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:54.756183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e19140-14f6-41f4-a986-3fb5e945b36c admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:54.757077 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e19140-14f6-41f4-a986-3fb5e945b36c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:54.759094 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e19140-14f6-41f4-a986-3fb5e945b36c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:54.761154 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:54.789376 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 815/1664] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:54.790782 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2e6a7-1262-4e5e-804b-b6c94e259825 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:54.813029 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2e6a7-1262-4e5e-804b-b6c94e259825 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:54.845277 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b60fec-ec95-42ba-a300-61cb5583aa37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:54.846036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b60fec-ec95-42ba-a300-61cb5583aa37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:54.846322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b60fec-ec95-42ba-a300-61cb5583aa37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:54.846628 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b60fec-ec95-42ba-a300-61cb5583aa37 admin admin] PATH_INFO: `/v3/projects/9609419b77ee4a388ea83b34408d4591/users/185575822d8e4f27a9e949b9f8af7d9a/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:54.875589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b60fec-ec95-42ba-a300-61cb5583aa37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9609419b77ee4a388ea83b34408d4591, user_id=185575822d8e4f27a9e949b9f8af7d9a, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:54.878399 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b60fec-ec95-42ba-a300-61cb5583aa37 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:54.932532 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 850/1665] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:33:54 2022] PUT /identity/v3/projects/9609419b77ee4a388ea83b34408d4591/users/185575822d8e4f27a9e949b9f8af7d9a/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:54.934742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc37990-ddd1-4309-8f4b-23fbac01311b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:54.935212 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc37990-ddd1-4309-8f4b-23fbac01311b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:54.935290 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc37990-ddd1-4309-8f4b-23fbac01311b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:54.995590 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7cc37990-ddd1-4309-8f4b-23fbac01311b None None] MFA Rules not processed for user `185575822d8e4f27a9e949b9f8af7d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:54.997482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7cc37990-ddd1-4309-8f4b-23fbac01311b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:55.260812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 816/1666] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:55.272870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b60fec-ec95-42ba-a300-61cb5583aa37 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:55.274727 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-09b60fec-ec95-42ba-a300-61cb5583aa37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:55.402852 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:55.408856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b60fec-ec95-42ba-a300-61cb5583aa37 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:55.415868 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1816c08-b4ae-4abc-aeb9-6ff9828c270c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:55.416799 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1816c08-b4ae-4abc-aeb9-6ff9828c270c service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:55.417529 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1816c08-b4ae-4abc-aeb9-6ff9828c270c service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:55.418004 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1816c08-b4ae-4abc-aeb9-6ff9828c270c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:55.444957 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1816c08-b4ae-4abc-aeb9-6ff9828c270c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:55.449044 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1816c08-b4ae-4abc-aeb9-6ff9828c270c service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:55.473820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 851/1667] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:55.588424 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc37990-ddd1-4309-8f4b-23fbac01311b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:55.814496 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc37990-ddd1-4309-8f4b-23fbac01311b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:55.832912 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfedb3-c519-4692-b213-574cccf7861c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:55.834337 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfedb3-c519-4692-b213-574cccf7861c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:55.834834 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfedb3-c519-4692-b213-574cccf7861c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:55.835326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfedb3-c519-4692-b213-574cccf7861c admin admin] PATH_INFO: `/v3/users/0faa07531d7f4360bfea8673650b0083` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:55.841296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dfedb3-c519-4692-b213-574cccf7861c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0faa07531d7f4360bfea8673650b0083)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:55.847853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dfedb3-c519-4692-b213-574cccf7861c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:56.003895 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-36dfedb3-c519-4692-b213-574cccf7861c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0faa07531d7f4360bfea8673650b0083'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:56.005608 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-36dfedb3-c519-4692-b213-574cccf7861c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0faa07531d7f4360bfea8673650b0083'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:56.018671 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-36dfedb3-c519-4692-b213-574cccf7861c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0faa07531d7f4360bfea8673650b0083'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:56.037359 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-36dfedb3-c519-4692-b213-574cccf7861c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0faa07531d7f4360bfea8673650b0083'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:56.038462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-36dfedb3-c519-4692-b213-574cccf7861c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0faa07531d7f4360bfea8673650b0083'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:56.049699 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:56.080986 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 817/1668] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:33:55 2022] DELETE /identity/v3/users/0faa07531d7f4360bfea8673650b0083 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:56.085823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1ecdb-e749-4f6e-8712-63e697080959 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:56.086369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1ecdb-e749-4f6e-8712-63e697080959 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:56.086832 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1ecdb-e749-4f6e-8712-63e697080959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:56.144645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-cbf1ecdb-e749-4f6e-8712-63e697080959 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:56.146352 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-cbf1ecdb-e749-4f6e-8712-63e697080959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:56.267019 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 852/1669] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:56.291056 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfedb3-c519-4692-b213-574cccf7861c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:56.292980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-36dfedb3-c519-4692-b213-574cccf7861c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:56.443576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfedb3-c519-4692-b213-574cccf7861c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:56.456455 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4d285-1b36-4075-9f64-cb2dcaca917a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:56.457456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4d285-1b36-4075-9f64-cb2dcaca917a service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:56.457669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4d285-1b36-4075-9f64-cb2dcaca917a service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:56.458046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4d285-1b36-4075-9f64-cb2dcaca917a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:56.482318 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4d285-1b36-4075-9f64-cb2dcaca917a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:56.485353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4d285-1b36-4075-9f64-cb2dcaca917a service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:56.502653 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 818/1670] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:33:57.683485 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1ecdb-e749-4f6e-8712-63e697080959 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:57.685030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-cbf1ecdb-e749-4f6e-8712-63e697080959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:57.717921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1ecdb-e749-4f6e-8712-63e697080959 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:57.739414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66b094-6497-4285-adf5-81d5908c47df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:57.740117 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66b094-6497-4285-adf5-81d5908c47df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:57.740394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66b094-6497-4285-adf5-81d5908c47df admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:57.740839 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66b094-6497-4285-adf5-81d5908c47df admin admin] PATH_INFO: `/v3/users/3bd2843f2b2a4aaeb39b9dd10b32351a` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:57.742576 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66b094-6497-4285-adf5-81d5908c47df admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bd2843f2b2a4aaeb39b9dd10b32351a)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:57.748018 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66b094-6497-4285-adf5-81d5908c47df admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:57.895714 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8b66b094-6497-4285-adf5-81d5908c47df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bd2843f2b2a4aaeb39b9dd10b32351a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:57.902253 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8b66b094-6497-4285-adf5-81d5908c47df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bd2843f2b2a4aaeb39b9dd10b32351a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:57.913441 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8b66b094-6497-4285-adf5-81d5908c47df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bd2843f2b2a4aaeb39b9dd10b32351a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:57.946291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8b66b094-6497-4285-adf5-81d5908c47df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bd2843f2b2a4aaeb39b9dd10b32351a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:57.964714 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8b66b094-6497-4285-adf5-81d5908c47df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bd2843f2b2a4aaeb39b9dd10b32351a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:57.984939 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 853/1671] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:33:57 2022] DELETE /identity/v3/users/3bd2843f2b2a4aaeb39b9dd10b32351a => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:57.986946 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebee5a-2946-4134-bd2f-b73b965a82e5 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:57.987442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebee5a-2946-4134-bd2f-b73b965a82e5 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:57.987620 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebee5a-2946-4134-bd2f-b73b965a82e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:58.035935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-71ebee5a-2946-4134-bd2f-b73b965a82e5 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:58.037229 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-71ebee5a-2946-4134-bd2f-b73b965a82e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:58.235372 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280442d3-fa8a-42c4-a030-4c89dc585355 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:58.235818 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280442d3-fa8a-42c4-a030-4c89dc585355 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:58.235920 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280442d3-fa8a-42c4-a030-4c89dc585355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:58.292657 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 819/1672] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:58.311357 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-280442d3-fa8a-42c4-a030-4c89dc585355 None None] MFA Rules not processed for user `ba149d1623e849328e399d6dbdca3e0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:58.314651 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-280442d3-fa8a-42c4-a030-4c89dc585355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:58.421748 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebee5a-2946-4134-bd2f-b73b965a82e5 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:58.426383 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-71ebee5a-2946-4134-bd2f-b73b965a82e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:58.468903 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:58.494645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 854/1673] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:33:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:33:58.518028 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280442d3-fa8a-42c4-a030-4c89dc585355 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:58.520037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-280442d3-fa8a-42c4-a030-4c89dc585355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:58.679701 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280442d3-fa8a-42c4-a030-4c89dc585355 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:58.684316 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebee5a-2946-4134-bd2f-b73b965a82e5 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:58.694350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a9696-4fac-4f3e-be3e-77bfe082adce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:58.695191 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a9696-4fac-4f3e-be3e-77bfe082adce service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:58.695575 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a9696-4fac-4f3e-be3e-77bfe082adce service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:58.695946 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a9696-4fac-4f3e-be3e-77bfe082adce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:58.702361 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e6446-47ed-464b-b3b4-cfff38e7912e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:58.712383 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e6446-47ed-464b-b3b4-cfff38e7912e service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:58.713600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e6446-47ed-464b-b3b4-cfff38e7912e service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:58.714146 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e6446-47ed-464b-b3b4-cfff38e7912e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:58.760955 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e6446-47ed-464b-b3b4-cfff38e7912e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:58.764489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e6446-47ed-464b-b3b4-cfff38e7912e service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:58.765659 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a9696-4fac-4f3e-be3e-77bfe082adce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:58.769991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a9696-4fac-4f3e-be3e-77bfe082adce service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:58.826861 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 820/1674] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:58.828947 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e6446-47ed-464b-b3b4-cfff38e7912e service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:58.834820 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-430e6446-47ed-464b-b3b4-cfff38e7912e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:58.839032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 855/1675] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:58.927604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e6446-47ed-464b-b3b4-cfff38e7912e service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:58.940737 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca68d35-4ec6-4721-aeab-d1da805f7353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:58.940991 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca68d35-4ec6-4721-aeab-d1da805f7353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:58.941452 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca68d35-4ec6-4721-aeab-d1da805f7353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:58.944917 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca68d35-4ec6-4721-aeab-d1da805f7353 admin admin] PATH_INFO: `/v3/projects/1d47773f079843f3a9d7d41b85a3b739` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:58.950984 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca68d35-4ec6-4721-aeab-d1da805f7353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d47773f079843f3a9d7d41b85a3b739)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:58.958928 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca68d35-4ec6-4721-aeab-d1da805f7353 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:59.098706 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:59.116186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fca68d35-4ec6-4721-aeab-d1da805f7353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d47773f079843f3a9d7d41b85a3b739', 'request_id': 'req-fca68d35-4ec6-4721-aeab-d1da805f7353'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.126836 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fca68d35-4ec6-4721-aeab-d1da805f7353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d47773f079843f3a9d7d41b85a3b739', 'request_id': 'req-fca68d35-4ec6-4721-aeab-d1da805f7353'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.145395 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fca68d35-4ec6-4721-aeab-d1da805f7353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d47773f079843f3a9d7d41b85a3b739', 'request_id': 'req-fca68d35-4ec6-4721-aeab-d1da805f7353'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.167589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fca68d35-4ec6-4721-aeab-d1da805f7353 admin admin] The token cache is being invalidate because project 1d47773f079843f3a9d7d41b85a3b739 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:33:59.168153 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fca68d35-4ec6-4721-aeab-d1da805f7353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.169362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fca68d35-4ec6-4721-aeab-d1da805f7353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.179856 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 821/1676] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:33:58 2022] DELETE /identity/v3/projects/1d47773f079843f3a9d7d41b85a3b739 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:59.181834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a9696-4fac-4f3e-be3e-77bfe082adce service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:59.208978 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a9696-4fac-4f3e-be3e-77bfe082adce service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:59.398406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21c96c-42f3-4312-b599-5d854b12befc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:59.399703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21c96c-42f3-4312-b599-5d854b12befc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:59.400469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21c96c-42f3-4312-b599-5d854b12befc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:59.401143 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21c96c-42f3-4312-b599-5d854b12befc admin admin] PATH_INFO: `/v3/users/185575822d8e4f27a9e949b9f8af7d9a` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:59.402715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21c96c-42f3-4312-b599-5d854b12befc admin admin] RBAC: Authorizing `identity:delete_user(user_id=185575822d8e4f27a9e949b9f8af7d9a)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:59.405694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21c96c-42f3-4312-b599-5d854b12befc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:59.551128 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-4d21c96c-42f3-4312-b599-5d854b12befc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '185575822d8e4f27a9e949b9f8af7d9a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.551822 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-4d21c96c-42f3-4312-b599-5d854b12befc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '185575822d8e4f27a9e949b9f8af7d9a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.576825 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-4d21c96c-42f3-4312-b599-5d854b12befc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '185575822d8e4f27a9e949b9f8af7d9a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.628177 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-4d21c96c-42f3-4312-b599-5d854b12befc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '185575822d8e4f27a9e949b9f8af7d9a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.649372 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-4d21c96c-42f3-4312-b599-5d854b12befc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '185575822d8e4f27a9e949b9f8af7d9a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.652243 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:59.669190 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 856/1677] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:33:59 2022] DELETE /identity/v3/users/185575822d8e4f27a9e949b9f8af7d9a => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:00.704301 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62239e-2d8a-467f-9f02-2bd3774dce64 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:00.705244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62239e-2d8a-467f-9f02-2bd3774dce64 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:00.705664 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62239e-2d8a-467f-9f02-2bd3774dce64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:00.792504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-8b62239e-2d8a-467f-9f02-2bd3774dce64 None None] MFA Rules not processed for user `c07d9c30b18a4150aa0bc7a1705551d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:00.795900 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8b62239e-2d8a-467f-9f02-2bd3774dce64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:01.017830 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 822/1678] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:01.028354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a08c7-269e-4845-b3f4-ecbdddcac37e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:01.028941 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a08c7-269e-4845-b3f4-ecbdddcac37e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:01.029681 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a08c7-269e-4845-b3f4-ecbdddcac37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:01.106834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-fd8a08c7-269e-4845-b3f4-ecbdddcac37e None None] MFA Rules not processed for user `c07d9c30b18a4150aa0bc7a1705551d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:01.111393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fd8a08c7-269e-4845-b3f4-ecbdddcac37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:01.139704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 857/1679] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:34:01.156316 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62239e-2d8a-467f-9f02-2bd3774dce64 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:01.158222 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8b62239e-2d8a-467f-9f02-2bd3774dce64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:01.362629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a08c7-269e-4845-b3f4-ecbdddcac37e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:01.366792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fd8a08c7-269e-4845-b3f4-ecbdddcac37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:01.372509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62239e-2d8a-467f-9f02-2bd3774dce64 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:01.396521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50395d27-95d4-48aa-9bfb-b9bbb9d247a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:01.397529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50395d27-95d4-48aa-9bfb-b9bbb9d247a6 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:01.397988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50395d27-95d4-48aa-9bfb-b9bbb9d247a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:01.399079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50395d27-95d4-48aa-9bfb-b9bbb9d247a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:01.463295 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50395d27-95d4-48aa-9bfb-b9bbb9d247a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:01.465261 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50395d27-95d4-48aa-9bfb-b9bbb9d247a6 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:01.513575 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 823/1680] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:34:01.636126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a08c7-269e-4845-b3f4-ecbdddcac37e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:01.648249 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b3bcc-4065-4849-be23-71a06b10afc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:01.654218 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b3bcc-4065-4849-be23-71a06b10afc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:01.654960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b3bcc-4065-4849-be23-71a06b10afc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:01.655411 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b3bcc-4065-4849-be23-71a06b10afc3 admin admin] PATH_INFO: `/v3/users/d8d79401b625483b818a017d53b8fe40` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:01.657396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b3bcc-4065-4849-be23-71a06b10afc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8d79401b625483b818a017d53b8fe40)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:01.660457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b3bcc-4065-4849-be23-71a06b10afc3 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:01.797123 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-643b3bcc-4065-4849-be23-71a06b10afc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8d79401b625483b818a017d53b8fe40'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:01.801572 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-643b3bcc-4065-4849-be23-71a06b10afc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8d79401b625483b818a017d53b8fe40'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:01.826464 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-643b3bcc-4065-4849-be23-71a06b10afc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8d79401b625483b818a017d53b8fe40'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:01.866378 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-643b3bcc-4065-4849-be23-71a06b10afc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8d79401b625483b818a017d53b8fe40'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:01.893048 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-643b3bcc-4065-4849-be23-71a06b10afc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8d79401b625483b818a017d53b8fe40'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:01.919438 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 858/1681] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:34:01 2022] DELETE /identity/v3/users/d8d79401b625483b818a017d53b8fe40 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:01.926280 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0b9dd-e081-44d9-a375-ce7f19e9b40e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:01.926752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0b9dd-e081-44d9-a375-ce7f19e9b40e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:01.927175 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0b9dd-e081-44d9-a375-ce7f19e9b40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:02.015401 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-03b0b9dd-e081-44d9-a375-ce7f19e9b40e None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:02.018887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-03b0b9dd-e081-44d9-a375-ce7f19e9b40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:02.145133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:02.174330 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 824/1682] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:02.197807 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b3bcc-4065-4849-be23-71a06b10afc3 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:02.199449 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-643b3bcc-4065-4849-be23-71a06b10afc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:02.255359 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0b9dd-e081-44d9-a375-ce7f19e9b40e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:02.257533 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-03b0b9dd-e081-44d9-a375-ce7f19e9b40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:02.289693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0b9dd-e081-44d9-a375-ce7f19e9b40e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:02.305929 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc6f1a-e711-4d7b-8e22-86da9521d3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:02.306927 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc6f1a-e711-4d7b-8e22-86da9521d3e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:02.307429 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc6f1a-e711-4d7b-8e22-86da9521d3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:02.307944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc6f1a-e711-4d7b-8e22-86da9521d3e4 admin admin] PATH_INFO: `/v3/projects/3904f939fbfb43c280dd8e547f3d4604` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:02.310361 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc6f1a-e711-4d7b-8e22-86da9521d3e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3904f939fbfb43c280dd8e547f3d4604)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:02.319267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc6f1a-e711-4d7b-8e22-86da9521d3e4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:02.355199 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b3bcc-4065-4849-be23-71a06b10afc3 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:02.373287 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7f138-0d86-441f-83f2-57e984a259a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:02.374050 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7f138-0d86-441f-83f2-57e984a259a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:02.374382 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7f138-0d86-441f-83f2-57e984a259a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:02.374661 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7f138-0d86-441f-83f2-57e984a259a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:02.402902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b7f138-0d86-441f-83f2-57e984a259a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:02.404563 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b7f138-0d86-441f-83f2-57e984a259a4 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:02.429770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 859/1683] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:02.454292 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-51fc6f1a-e711-4d7b-8e22-86da9521d3e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3904f939fbfb43c280dd8e547f3d4604', 'request_id': 'req-51fc6f1a-e711-4d7b-8e22-86da9521d3e4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:02.474035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-51fc6f1a-e711-4d7b-8e22-86da9521d3e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3904f939fbfb43c280dd8e547f3d4604', 'request_id': 'req-51fc6f1a-e711-4d7b-8e22-86da9521d3e4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:02.483183 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be3e85-c649-4e4e-ab53-096d53f1d2df None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:02.485424 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be3e85-c649-4e4e-ab53-096d53f1d2df None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:02.485756 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be3e85-c649-4e4e-ab53-096d53f1d2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:02.493043 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-51fc6f1a-e711-4d7b-8e22-86da9521d3e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3904f939fbfb43c280dd8e547f3d4604', 'request_id': 'req-51fc6f1a-e711-4d7b-8e22-86da9521d3e4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:02.535683 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-51fc6f1a-e711-4d7b-8e22-86da9521d3e4 admin admin] The token cache is being invalidate because project 3904f939fbfb43c280dd8e547f3d4604 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:02.535986 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-51fc6f1a-e711-4d7b-8e22-86da9521d3e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:02.538739 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-51fc6f1a-e711-4d7b-8e22-86da9521d3e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:02.543371 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 825/1684] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:02 2022] DELETE /identity/v3/projects/3904f939fbfb43c280dd8e547f3d4604 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:02.551493 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d68150-190a-4869-88f0-00abc10d07d9 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:02.551794 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d68150-190a-4869-88f0-00abc10d07d9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:02.552102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d68150-190a-4869-88f0-00abc10d07d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:02.596658 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:02.605786 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a2be3e85-c649-4e4e-ab53-096d53f1d2df None None] MFA Rules not processed for user `7d1b6c388c7642a892c905d16f3b8d50`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:02.608781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a2be3e85-c649-4e4e-ab53-096d53f1d2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:02.637531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f4d68150-190a-4869-88f0-00abc10d07d9 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:02.644220 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f4d68150-190a-4869-88f0-00abc10d07d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:02.791981 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 860/1685] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:34:02.792889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be3e85-c649-4e4e-ab53-096d53f1d2df None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:02.802244 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a2be3e85-c649-4e4e-ab53-096d53f1d2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:02.926742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 826/1686] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:02.928427 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d68150-190a-4869-88f0-00abc10d07d9 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:02.930409 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f4d68150-190a-4869-88f0-00abc10d07d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:02.958938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be3e85-c649-4e4e-ab53-096d53f1d2df None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:02.965033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662032f0-c67f-47b1-8028-2ce89f7f2584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:02.965728 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662032f0-c67f-47b1-8028-2ce89f7f2584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:02.966041 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662032f0-c67f-47b1-8028-2ce89f7f2584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:02.966321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662032f0-c67f-47b1-8028-2ce89f7f2584 admin admin] PATH_INFO: `/v3/projects/9609419b77ee4a388ea83b34408d4591` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:02.983955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662032f0-c67f-47b1-8028-2ce89f7f2584 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9609419b77ee4a388ea83b34408d4591)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:02.985023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: WARNING py.warnings [None req-662032f0-c67f-47b1-8028-2ce89f7f2584 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:34:02.985023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: warnings.warn(msg) Aug 01 15:34:02.985023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: Aug 01 15:34:02.986289 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662032f0-c67f-47b1-8028-2ce89f7f2584 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:03.119551 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-662032f0-c67f-47b1-8028-2ce89f7f2584 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9609419b77ee4a388ea83b34408d4591', 'request_id': 'req-662032f0-c67f-47b1-8028-2ce89f7f2584'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:03.130383 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-662032f0-c67f-47b1-8028-2ce89f7f2584 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9609419b77ee4a388ea83b34408d4591', 'request_id': 'req-662032f0-c67f-47b1-8028-2ce89f7f2584'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:03.167025 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-662032f0-c67f-47b1-8028-2ce89f7f2584 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9609419b77ee4a388ea83b34408d4591', 'request_id': 'req-662032f0-c67f-47b1-8028-2ce89f7f2584'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:03.179292 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d68150-190a-4869-88f0-00abc10d07d9 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:03.190021 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-662032f0-c67f-47b1-8028-2ce89f7f2584 admin admin] The token cache is being invalidate because project 9609419b77ee4a388ea83b34408d4591 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:03.190021 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-662032f0-c67f-47b1-8028-2ce89f7f2584 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:03.190937 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-662032f0-c67f-47b1-8028-2ce89f7f2584 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:03.193071 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:03.196884 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 861/1687] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:02 2022] DELETE /identity/v3/projects/9609419b77ee4a388ea83b34408d4591 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:03.198768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc76a35-e0ae-46d7-b9f6-8cf546670a95 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:03.198768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc76a35-e0ae-46d7-b9f6-8cf546670a95 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:03.199510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc76a35-e0ae-46d7-b9f6-8cf546670a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:03.246697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-3dc76a35-e0ae-46d7-b9f6-8cf546670a95 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:03.246697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3dc76a35-e0ae-46d7-b9f6-8cf546670a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:03.317027 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4b5b7-57c1-4300-af73-777d16a23e60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:03.317751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4b5b7-57c1-4300-af73-777d16a23e60 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:03.319144 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4b5b7-57c1-4300-af73-777d16a23e60 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:03.319445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4b5b7-57c1-4300-af73-777d16a23e60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:03.320806 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:03.324608 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-39d4b5b7-57c1-4300-af73-777d16a23e60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:03.399867 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 862/1688] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:03.401463 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63f8c3-a1b5-4e41-963b-af336b2079dd None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:03.401929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63f8c3-a1b5-4e41-963b-af336b2079dd None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:03.402367 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63f8c3-a1b5-4e41-963b-af336b2079dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:03.441789 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8e63f8c3-a1b5-4e41-963b-af336b2079dd None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:03.443471 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8e63f8c3-a1b5-4e41-963b-af336b2079dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:03.455159 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 863/1689] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:03.457745 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbaf67-b6d3-427c-addd-de59d0d091cc None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:03.458702 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbaf67-b6d3-427c-addd-de59d0d091cc None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:03.459343 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbaf67-b6d3-427c-addd-de59d0d091cc None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:03.471857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 864/1690] 10.222.0.83 () {54 vars in 912 bytes} [Mon Aug 1 15:34:03 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:34:03.473965 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbaf67-b6d3-427c-addd-de59d0d091cc None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:03.506657 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbaf67-b6d3-427c-addd-de59d0d091cc None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:03.516028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4b5b7-57c1-4300-af73-777d16a23e60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:03.516880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4c8c6-ed5d-4e40-b9bb-4bc4fa672d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:03.517974 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4c8c6-ed5d-4e40-b9bb-4bc4fa672d13 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:03.518516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4c8c6-ed5d-4e40-b9bb-4bc4fa672d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:03.522552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4c8c6-ed5d-4e40-b9bb-4bc4fa672d13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:03.523805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4b5b7-57c1-4300-af73-777d16a23e60 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:03.525446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4c8c6-ed5d-4e40-b9bb-4bc4fa672d13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:34:03.526502 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4c8c6-ed5d-4e40-b9bb-4bc4fa672d13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:03.531048 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4c8c6-ed5d-4e40-b9bb-4bc4fa672d13 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:03.555026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 827/1691] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:03.559952 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea98c56-0f73-453e-9333-65529b654519 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:03.560637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea98c56-0f73-453e-9333-65529b654519 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:03.560637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea98c56-0f73-453e-9333-65529b654519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:03.569077 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 865/1692] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:34:03.572610 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d6c35-15f0-453e-9018-5a03ea29dcaa None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:03.574502 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d6c35-15f0-453e-9018-5a03ea29dcaa None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:03.575071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d6c35-15f0-453e-9018-5a03ea29dcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:03.626575 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3ea98c56-0f73-453e-9333-65529b654519 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:03.630595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3ea98c56-0f73-453e-9333-65529b654519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:03.656170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 828/1693] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:03.658224 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea98c56-0f73-453e-9333-65529b654519 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:03.661764 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3ea98c56-0f73-453e-9333-65529b654519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:03.673666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-fa9d6c35-15f0-453e-9018-5a03ea29dcaa None None] MFA Rules not processed for user `76311c7173b442c0af2c745e3e70a07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:03.678532 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fa9d6c35-15f0-453e-9018-5a03ea29dcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:03.715025 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea98c56-0f73-453e-9333-65529b654519 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:03.723742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31da2b5-23a3-40d1-8e37-52580ef65fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:03.724430 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31da2b5-23a3-40d1-8e37-52580ef65fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:03.724719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31da2b5-23a3-40d1-8e37-52580ef65fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:03.724999 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31da2b5-23a3-40d1-8e37-52580ef65fb4 admin admin] PATH_INFO: `/v3/projects/04503417fb334760a73bded842054784` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:03.727234 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31da2b5-23a3-40d1-8e37-52580ef65fb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04503417fb334760a73bded842054784)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:03.729209 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31da2b5-23a3-40d1-8e37-52580ef65fb4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:03.833846 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 866/1694] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Aug 1 15:34:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:03.835100 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d6c35-15f0-453e-9018-5a03ea29dcaa None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:03.840991 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b31da2b5-23a3-40d1-8e37-52580ef65fb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04503417fb334760a73bded842054784', 'request_id': 'req-b31da2b5-23a3-40d1-8e37-52580ef65fb4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:03.857510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b31da2b5-23a3-40d1-8e37-52580ef65fb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04503417fb334760a73bded842054784', 'request_id': 'req-b31da2b5-23a3-40d1-8e37-52580ef65fb4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:03.863694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d6c35-15f0-453e-9018-5a03ea29dcaa None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:03.875191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b31da2b5-23a3-40d1-8e37-52580ef65fb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04503417fb334760a73bded842054784', 'request_id': 'req-b31da2b5-23a3-40d1-8e37-52580ef65fb4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:03.910478 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b31da2b5-23a3-40d1-8e37-52580ef65fb4 admin admin] The token cache is being invalidate because project 04503417fb334760a73bded842054784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:03.910798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b31da2b5-23a3-40d1-8e37-52580ef65fb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:03.911691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b31da2b5-23a3-40d1-8e37-52580ef65fb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:03.916169 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 829/1695] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:03 2022] DELETE /identity/v3/projects/04503417fb334760a73bded842054784 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:03.917231 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31da2b5-23a3-40d1-8e37-52580ef65fb4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:03.930250 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b31da2b5-23a3-40d1-8e37-52580ef65fb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:03.983101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:04.038390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17494fc-839e-4c3f-b7ca-14bd3720ec0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:04.040054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17494fc-839e-4c3f-b7ca-14bd3720ec0e admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:04.042193 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17494fc-839e-4c3f-b7ca-14bd3720ec0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:04.042535 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17494fc-839e-4c3f-b7ca-14bd3720ec0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:04.052158 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17494fc-839e-4c3f-b7ca-14bd3720ec0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:04.058247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17494fc-839e-4c3f-b7ca-14bd3720ec0e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:04.114056 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31da2b5-23a3-40d1-8e37-52580ef65fb4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:04.127918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 867/1696] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:34:03 2022] POST /identity/v3/projects => generated 403 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:04.128156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cac2fc-1420-48af-97e8-9975ea9c50db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:04.128884 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17494fc-839e-4c3f-b7ca-14bd3720ec0e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:04.129162 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cac2fc-1420-48af-97e8-9975ea9c50db admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:04.129597 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cac2fc-1420-48af-97e8-9975ea9c50db admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:04.130222 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cac2fc-1420-48af-97e8-9975ea9c50db admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:04.131501 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cac2fc-1420-48af-97e8-9975ea9c50db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:34:04.133482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cac2fc-1420-48af-97e8-9975ea9c50db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:04.137916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cac2fc-1420-48af-97e8-9975ea9c50db admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:04.143655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d17494fc-839e-4c3f-b7ca-14bd3720ec0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:04.193077 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 830/1697] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:34:04.197605 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cac2fc-1420-48af-97e8-9975ea9c50db admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:04.210564 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-07cac2fc-1420-48af-97e8-9975ea9c50db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:04.227831 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:04.243665 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cac2fc-1420-48af-97e8-9975ea9c50db admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:04.278346 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17494fc-839e-4c3f-b7ca-14bd3720ec0e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:04.296868 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511db847-93e3-4fd5-a9de-8dc4a3deb789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:04.297421 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511db847-93e3-4fd5-a9de-8dc4a3deb789 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:04.297725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511db847-93e3-4fd5-a9de-8dc4a3deb789 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:04.298017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511db847-93e3-4fd5-a9de-8dc4a3deb789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:04.310736 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-511db847-93e3-4fd5-a9de-8dc4a3deb789 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:04.325590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511db847-93e3-4fd5-a9de-8dc4a3deb789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:04.327658 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511db847-93e3-4fd5-a9de-8dc4a3deb789 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:04.362920 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 868/1698] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:04.368056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511db847-93e3-4fd5-a9de-8dc4a3deb789 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:04.372393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-511db847-93e3-4fd5-a9de-8dc4a3deb789 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:04.398303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8406a582-db6a-4864-8bac-31b773cd07bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:04.399197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406a582-db6a-4864-8bac-31b773cd07bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:04.399704 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406a582-db6a-4864-8bac-31b773cd07bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:04.400205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406a582-db6a-4864-8bac-31b773cd07bb admin admin] PATH_INFO: `/v3/users/3cad08ce5bc04bc0ad6a1feb606f2935` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:04.402161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8406a582-db6a-4864-8bac-31b773cd07bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cad08ce5bc04bc0ad6a1feb606f2935)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:04.415266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8406a582-db6a-4864-8bac-31b773cd07bb admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:04.499734 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8406a582-db6a-4864-8bac-31b773cd07bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cad08ce5bc04bc0ad6a1feb606f2935'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:04.500817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8406a582-db6a-4864-8bac-31b773cd07bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cad08ce5bc04bc0ad6a1feb606f2935'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:04.529690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8406a582-db6a-4864-8bac-31b773cd07bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cad08ce5bc04bc0ad6a1feb606f2935'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:04.554108 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511db847-93e3-4fd5-a9de-8dc4a3deb789 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:04.565091 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8406a582-db6a-4864-8bac-31b773cd07bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cad08ce5bc04bc0ad6a1feb606f2935'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:04.566730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8406a582-db6a-4864-8bac-31b773cd07bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cad08ce5bc04bc0ad6a1feb606f2935'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:04.573533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fc5d4-c9bf-4923-a0cd-681206e67f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:04.580083 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fc5d4-c9bf-4923-a0cd-681206e67f1f admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:04.581023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fc5d4-c9bf-4923-a0cd-681206e67f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:04.581511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fc5d4-c9bf-4923-a0cd-681206e67f1f admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:04.586156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fc5d4-c9bf-4923-a0cd-681206e67f1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:04.595957 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fc5d4-c9bf-4923-a0cd-681206e67f1f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:04.604595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 831/1699] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:34:04 2022] DELETE /identity/v3/users/3cad08ce5bc04bc0ad6a1feb606f2935 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:04.607382 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8406a582-db6a-4864-8bac-31b773cd07bb admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:04.612366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8406a582-db6a-4864-8bac-31b773cd07bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:04.651414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 869/1700] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:34:04 2022] POST /identity/v3/users => generated 302 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:04.658884 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fc5d4-c9bf-4923-a0cd-681206e67f1f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:04.663628 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-174fc5d4-c9bf-4923-a0cd-681206e67f1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:04.681054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fc5d4-c9bf-4923-a0cd-681206e67f1f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:04.710281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:04.765347 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8406a582-db6a-4864-8bac-31b773cd07bb admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:04.774421 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb514b2d-c477-4bf9-af94-e895480d8610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:04.775522 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb514b2d-c477-4bf9-af94-e895480d8610 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:04.775973 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb514b2d-c477-4bf9-af94-e895480d8610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:04.776401 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb514b2d-c477-4bf9-af94-e895480d8610 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:04.777253 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb514b2d-c477-4bf9-af94-e895480d8610 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:04.779531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb514b2d-c477-4bf9-af94-e895480d8610 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:04.797051 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ae536-9f08-4f98-953c-bcf58135db20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:04.804658 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ae536-9f08-4f98-953c-bcf58135db20 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:04.804962 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ae536-9f08-4f98-953c-bcf58135db20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:04.805637 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ae536-9f08-4f98-953c-bcf58135db20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:04.806867 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ae536-9f08-4f98-953c-bcf58135db20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:04.810705 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ae536-9f08-4f98-953c-bcf58135db20 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:04.840421 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 870/1701] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:04.844690 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ae536-9f08-4f98-953c-bcf58135db20 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:04.846102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 832/1702] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:34:04 2022] POST /identity/v3/projects => generated 381 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:04.854338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb514b2d-c477-4bf9-af94-e895480d8610 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:04.860740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ae536-9f08-4f98-953c-bcf58135db20 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:05.039651 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5704a-92bb-4fd4-b055-979b3cff16a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:05.040160 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5704a-92bb-4fd4-b055-979b3cff16a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:05.040549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5704a-92bb-4fd4-b055-979b3cff16a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:05.040855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5704a-92bb-4fd4-b055-979b3cff16a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:05.041596 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5704a-92bb-4fd4-b055-979b3cff16a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:05.043270 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5704a-92bb-4fd4-b055-979b3cff16a9 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:05.050080 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb514b2d-c477-4bf9-af94-e895480d8610 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:05.058873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13265716-284b-45f4-a45c-04f93b73fc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:05.059462 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:05.059642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13265716-284b-45f4-a45c-04f93b73fc9b admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:05.059878 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13265716-284b-45f4-a45c-04f93b73fc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:05.060100 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13265716-284b-45f4-a45c-04f93b73fc9b admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:05.060910 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13265716-284b-45f4-a45c-04f93b73fc9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:05.063681 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13265716-284b-45f4-a45c-04f93b73fc9b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:05.075468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 871/1703] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:05.079959 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5704a-92bb-4fd4-b055-979b3cff16a9 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:05.099071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5704a-92bb-4fd4-b055-979b3cff16a9 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:05.106870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137e3e3-9b2f-41c1-88f1-82ed89d4d858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:05.107625 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137e3e3-9b2f-41c1-88f1-82ed89d4d858 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:05.108537 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137e3e3-9b2f-41c1-88f1-82ed89d4d858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:05.108825 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137e3e3-9b2f-41c1-88f1-82ed89d4d858 admin admin] PATH_INFO: `/v3/projects/a7936102185d4a7f994d337368487582/users/79cefb8ac83f470e86d2f28845854d18/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:05.114234 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 833/1704] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:34:04 2022] POST /identity/v3/users => generated 291 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:05.123033 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13265716-284b-45f4-a45c-04f93b73fc9b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:05.146254 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13265716-284b-45f4-a45c-04f93b73fc9b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:05.153928 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3132df-56ff-44a6-96cc-d8588f80c6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:05.155364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3132df-56ff-44a6-96cc-d8588f80c6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:05.155364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3132df-56ff-44a6-96cc-d8588f80c6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:05.155668 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3132df-56ff-44a6-96cc-d8588f80c6f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:05.156356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3132df-56ff-44a6-96cc-d8588f80c6f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:05.158081 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3132df-56ff-44a6-96cc-d8588f80c6f8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:05.167392 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5137e3e3-9b2f-41c1-88f1-82ed89d4d858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7936102185d4a7f994d337368487582, user_id=79cefb8ac83f470e86d2f28845854d18, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:05.170167 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5137e3e3-9b2f-41c1-88f1-82ed89d4d858 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:05.192205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 834/1705] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:05.196342 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3132df-56ff-44a6-96cc-d8588f80c6f8 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:05.203105 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 872/1706] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:05 2022] PUT /identity/v3/projects/a7936102185d4a7f994d337368487582/users/79cefb8ac83f470e86d2f28845854d18/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:05.208069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d3a1c-007e-4d01-a334-650249f69bd2 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:05.208691 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d3a1c-007e-4d01-a334-650249f69bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:05.208875 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d3a1c-007e-4d01-a334-650249f69bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:05.220281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3132df-56ff-44a6-96cc-d8588f80c6f8 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:05.260096 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-525d3a1c-007e-4d01-a334-650249f69bd2 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:05.261345 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-525d3a1c-007e-4d01-a334-650249f69bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:05.366160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939a0bd-03ca-427d-bffd-b8028e78ba07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:05.366833 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939a0bd-03ca-427d-bffd-b8028e78ba07 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:05.367119 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939a0bd-03ca-427d-bffd-b8028e78ba07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:05.367405 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939a0bd-03ca-427d-bffd-b8028e78ba07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:05.368137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939a0bd-03ca-427d-bffd-b8028e78ba07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:05.370172 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939a0bd-03ca-427d-bffd-b8028e78ba07 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:05.392939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 835/1707] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:34:05.397363 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939a0bd-03ca-427d-bffd-b8028e78ba07 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:05.417053 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 873/1708] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:34:05.422749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939a0bd-03ca-427d-bffd-b8028e78ba07 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:05.441866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8bee7b-b3db-483c-ae08-09be37905958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:05.442860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bee7b-b3db-483c-ae08-09be37905958 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:05.443298 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bee7b-b3db-483c-ae08-09be37905958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:05.443707 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bee7b-b3db-483c-ae08-09be37905958 admin admin] PATH_INFO: `/v3/projects/9b6b52540e8e4b5a9db60b4bcbc227fc/users/ccf5a2b5a3f34f7285f437ef2b9a94a1/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:05.496849 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8bee7b-b3db-483c-ae08-09be37905958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b6b52540e8e4b5a9db60b4bcbc227fc, user_id=ccf5a2b5a3f34f7285f437ef2b9a94a1, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:05.500542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8bee7b-b3db-483c-ae08-09be37905958 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:05.542307 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d3a1c-007e-4d01-a334-650249f69bd2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:05.543822 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-525d3a1c-007e-4d01-a334-650249f69bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:05.610558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 836/1709] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:05 2022] PUT /identity/v3/projects/9b6b52540e8e4b5a9db60b4bcbc227fc/users/ccf5a2b5a3f34f7285f437ef2b9a94a1/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:05.614964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2b056-7866-482e-92dd-b419b1f98f49 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:05.622072 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2b056-7866-482e-92dd-b419b1f98f49 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:05.622759 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2b056-7866-482e-92dd-b419b1f98f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:05.758832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:05.763029 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d3a1c-007e-4d01-a334-650249f69bd2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:05.786394 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-dfe2b056-7866-482e-92dd-b419b1f98f49 None None] MFA Rules not processed for user `ccf5a2b5a3f34f7285f437ef2b9a94a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:05.787673 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-dfe2b056-7866-482e-92dd-b419b1f98f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:05.868576 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04617e-fc9a-4be3-9f21-459f5cad57a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:05.869081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04617e-fc9a-4be3-9f21-459f5cad57a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:05.869367 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04617e-fc9a-4be3-9f21-459f5cad57a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:05.869643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04617e-fc9a-4be3-9f21-459f5cad57a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:05.976242 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 837/1710] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:05.980667 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2b056-7866-482e-92dd-b419b1f98f49 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:05.999329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:06.053557 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04617e-fc9a-4be3-9f21-459f5cad57a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:06.058890 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04617e-fc9a-4be3-9f21-459f5cad57a4 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:06.087726 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 874/1711] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:06.166654 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2b056-7866-482e-92dd-b419b1f98f49 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:06.185879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbb123-06f1-4711-8fff-8c87fba5f0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:06.186579 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbb123-06f1-4711-8fff-8c87fba5f0f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:06.186858 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbb123-06f1-4711-8fff-8c87fba5f0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:06.187131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbb123-06f1-4711-8fff-8c87fba5f0f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:06.187817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbbb123-06f1-4711-8fff-8c87fba5f0f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:06.190167 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbbb123-06f1-4711-8fff-8c87fba5f0f1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:06.222496 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12719de-953e-4b74-b503-09b176bb9604 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:06.222794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12719de-953e-4b74-b503-09b176bb9604 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:06.223072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12719de-953e-4b74-b503-09b176bb9604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:06.256032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 838/1712] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:34:05 2022] POST /identity/v3/projects => generated 381 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:06.260926 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbb123-06f1-4711-8fff-8c87fba5f0f1 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:06.301254 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e12719de-953e-4b74-b503-09b176bb9604 None None] MFA Rules not processed for user `ba149d1623e849328e399d6dbdca3e0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:06.304082 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e12719de-953e-4b74-b503-09b176bb9604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:06.455223 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbb123-06f1-4711-8fff-8c87fba5f0f1 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:06.464897 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5319cc-0558-4bd0-89c9-ba58914eb483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:06.465554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5319cc-0558-4bd0-89c9-ba58914eb483 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:06.465847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5319cc-0558-4bd0-89c9-ba58914eb483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:06.466206 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5319cc-0558-4bd0-89c9-ba58914eb483 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:06.466912 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5319cc-0558-4bd0-89c9-ba58914eb483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:06.469482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5319cc-0558-4bd0-89c9-ba58914eb483 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:06.534502 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 839/1713] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:34:06 2022] POST /identity/v3/users => generated 297 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:34:06.536161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8fbcd-e57c-4e9f-a087-a3a3136c3ea0 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:06.536462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8fbcd-e57c-4e9f-a087-a3a3136c3ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:06.536753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8fbcd-e57c-4e9f-a087-a3a3136c3ea0 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:06.543335 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 840/1714] 10.222.0.83 () {54 vars in 961 bytes} [Mon Aug 1 15:34:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:34:06.543941 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8fbcd-e57c-4e9f-a087-a3a3136c3ea0 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:06.552323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 875/1715] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:34:06.557901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e883d6-e89f-4f92-9c16-605b0dea0410 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:06.558200 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e883d6-e89f-4f92-9c16-605b0dea0410 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:06.558483 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e883d6-e89f-4f92-9c16-605b0dea0410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:06.599137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8fbcd-e57c-4e9f-a087-a3a3136c3ea0 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:06.620832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeae8f6-448d-4f83-bed5-366dd91a47bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:06.621495 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeae8f6-448d-4f83-bed5-366dd91a47bd admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:06.621787 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeae8f6-448d-4f83-bed5-366dd91a47bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:06.622097 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeae8f6-448d-4f83-bed5-366dd91a47bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:06.622819 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeae8f6-448d-4f83-bed5-366dd91a47bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:06.625017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeae8f6-448d-4f83-bed5-366dd91a47bd admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:06.652092 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 841/1716] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:06.662454 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeae8f6-448d-4f83-bed5-366dd91a47bd admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:06.670328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:06.678988 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-61e883d6-e89f-4f92-9c16-605b0dea0410 None None] MFA Rules not processed for user `ef57a9394b114eb3ab499133b5d40781`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:06.680306 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-61e883d6-e89f-4f92-9c16-605b0dea0410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:06.684434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeae8f6-448d-4f83-bed5-366dd91a47bd admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:06.710100 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc2d88-5afd-4f55-9469-9a9a05f8dd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:06.713688 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc2d88-5afd-4f55-9469-9a9a05f8dd67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:06.714822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc2d88-5afd-4f55-9469-9a9a05f8dd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:06.715825 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc2d88-5afd-4f55-9469-9a9a05f8dd67 admin admin] PATH_INFO: `/v3/projects/4789ef07a02a4cf2a77339ebaf0b254d/users/936ff0fe3cb3430aab4474478197b0b6/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:06.780917 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc2d88-5afd-4f55-9469-9a9a05f8dd67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4789ef07a02a4cf2a77339ebaf0b254d, user_id=936ff0fe3cb3430aab4474478197b0b6, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:06.790433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc2d88-5afd-4f55-9469-9a9a05f8dd67 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:06.828975 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 842/1717] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:06 2022] PUT /identity/v3/projects/4789ef07a02a4cf2a77339ebaf0b254d/users/936ff0fe3cb3430aab4474478197b0b6/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:06.832881 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc2d88-5afd-4f55-9469-9a9a05f8dd67 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:06.900442 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 876/1718] 10.222.0.83 () {58 vars in 1058 bytes} [Mon Aug 1 15:34:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:06.907030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e883d6-e89f-4f92-9c16-605b0dea0410 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:06.931935 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e883d6-e89f-4f92-9c16-605b0dea0410 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:07.011162 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc2d88-5afd-4f55-9469-9a9a05f8dd67 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:07.019941 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ba727-a02d-4fb0-bf5d-f185facd6fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:07.021043 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ba727-a02d-4fb0-bf5d-f185facd6fef admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:07.021043 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ba727-a02d-4fb0-bf5d-f185facd6fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:07.021324 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ba727-a02d-4fb0-bf5d-f185facd6fef admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:07.021932 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ba727-a02d-4fb0-bf5d-f185facd6fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:07.023587 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ba727-a02d-4fb0-bf5d-f185facd6fef admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:07.054903 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 843/1719] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:07.058737 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ba727-a02d-4fb0-bf5d-f185facd6fef admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:07.087167 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ba727-a02d-4fb0-bf5d-f185facd6fef admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:07.100092 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc7011-94ec-4fcb-8f5f-7055e2dce7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:07.100729 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc7011-94ec-4fcb-8f5f-7055e2dce7b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:07.101021 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc7011-94ec-4fcb-8f5f-7055e2dce7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:07.101301 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc7011-94ec-4fcb-8f5f-7055e2dce7b2 admin admin] PATH_INFO: `/v3/projects/4789ef07a02a4cf2a77339ebaf0b254d/users/936ff0fe3cb3430aab4474478197b0b6/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:07.105785 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc7011-94ec-4fcb-8f5f-7055e2dce7b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4789ef07a02a4cf2a77339ebaf0b254d, user_id=936ff0fe3cb3430aab4474478197b0b6, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:07.111680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc7011-94ec-4fcb-8f5f-7055e2dce7b2 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:07.145242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79283541-b013-45dd-84dc-90b3956af302 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:07.148529 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79283541-b013-45dd-84dc-90b3956af302 service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:07.149453 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79283541-b013-45dd-84dc-90b3956af302 service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:07.150282 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79283541-b013-45dd-84dc-90b3956af302 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:07.159987 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-79283541-b013-45dd-84dc-90b3956af302 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:07.174906 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 844/1720] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:07 2022] PUT /identity/v3/projects/4789ef07a02a4cf2a77339ebaf0b254d/users/936ff0fe3cb3430aab4474478197b0b6/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:34:07.178228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79283541-b013-45dd-84dc-90b3956af302 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:07.181005 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79283541-b013-45dd-84dc-90b3956af302 service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:07.188862 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429be3c8-5fd6-4b71-961c-e7e0d2a0dd59 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:07.189137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429be3c8-5fd6-4b71-961c-e7e0d2a0dd59 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:07.189491 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429be3c8-5fd6-4b71-961c-e7e0d2a0dd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:07.230917 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 877/1721] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:34:07.277933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-429be3c8-5fd6-4b71-961c-e7e0d2a0dd59 None None] MFA Rules not processed for user `936ff0fe3cb3430aab4474478197b0b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:07.279223 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-429be3c8-5fd6-4b71-961c-e7e0d2a0dd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:07.315078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79283541-b013-45dd-84dc-90b3956af302 service placement] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:07.317336 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-79283541-b013-45dd-84dc-90b3956af302 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:07.349567 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79283541-b013-45dd-84dc-90b3956af302 service placement] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:07.394364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:07.503765 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 845/1722] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:07.566446 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429be3c8-5fd6-4b71-961c-e7e0d2a0dd59 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:07.575697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-429be3c8-5fd6-4b71-961c-e7e0d2a0dd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:07.613630 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98bea4-717b-4d77-9a24-6f12d1ace9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:07.614345 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98bea4-717b-4d77-9a24-6f12d1ace9f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:07.614625 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98bea4-717b-4d77-9a24-6f12d1ace9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:07.614894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98bea4-717b-4d77-9a24-6f12d1ace9f6 admin admin] PATH_INFO: `/v3/projects/f092da46cb5748439ca4c0ee8dda6489` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:07.617925 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d98bea4-717b-4d77-9a24-6f12d1ace9f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f092da46cb5748439ca4c0ee8dda6489)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:07.624709 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d98bea4-717b-4d77-9a24-6f12d1ace9f6 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:07.787290 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:07.815107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429be3c8-5fd6-4b71-961c-e7e0d2a0dd59 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:07.826074 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5d98bea4-717b-4d77-9a24-6f12d1ace9f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f092da46cb5748439ca4c0ee8dda6489', 'request_id': 'req-5d98bea4-717b-4d77-9a24-6f12d1ace9f6'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:07.846585 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5d98bea4-717b-4d77-9a24-6f12d1ace9f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f092da46cb5748439ca4c0ee8dda6489', 'request_id': 'req-5d98bea4-717b-4d77-9a24-6f12d1ace9f6'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:07.879395 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5d98bea4-717b-4d77-9a24-6f12d1ace9f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f092da46cb5748439ca4c0ee8dda6489', 'request_id': 'req-5d98bea4-717b-4d77-9a24-6f12d1ace9f6'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:07.923064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5d98bea4-717b-4d77-9a24-6f12d1ace9f6 admin admin] The token cache is being invalidate because project f092da46cb5748439ca4c0ee8dda6489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:07.923381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5d98bea4-717b-4d77-9a24-6f12d1ace9f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:07.924479 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5d98bea4-717b-4d77-9a24-6f12d1ace9f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:07.936711 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 878/1723] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:07 2022] DELETE /identity/v3/projects/f092da46cb5748439ca4c0ee8dda6489 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:34:07.949717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44750ac-89ea-4f01-8578-137d59ec9f30 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:07.950066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44750ac-89ea-4f01-8578-137d59ec9f30 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:07.951844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44750ac-89ea-4f01-8578-137d59ec9f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:08.049085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a44750ac-89ea-4f01-8578-137d59ec9f30 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:08.055808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a44750ac-89ea-4f01-8578-137d59ec9f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:08.056505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9b7db-adb1-46ae-9528-393943d20c40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:08.057192 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9b7db-adb1-46ae-9528-393943d20c40 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:08.057621 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9b7db-adb1-46ae-9528-393943d20c40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:08.058086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9b7db-adb1-46ae-9528-393943d20c40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:08.082336 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-48f9b7db-adb1-46ae-9528-393943d20c40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:08.332527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 879/1724] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:08.334420 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f604b-d57c-4429-9a7b-f80b805a7cbc None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:08.334714 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f604b-d57c-4429-9a7b-f80b805a7cbc None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:08.337686 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f604b-d57c-4429-9a7b-f80b805a7cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:08.347846 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9b7db-adb1-46ae-9528-393943d20c40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:08.349360 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9b7db-adb1-46ae-9528-393943d20c40 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:08.377407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 846/1725] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 838 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:08.428540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-156f604b-d57c-4429-9a7b-f80b805a7cbc None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:08.432941 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-156f604b-d57c-4429-9a7b-f80b805a7cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:08.491959 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 880/1726] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:08.497021 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9b7db-adb1-46ae-9528-393943d20c40 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:08.527581 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9b7db-adb1-46ae-9528-393943d20c40 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:08.541362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccefcd7-7335-4430-9c3a-0b56d0490cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:08.542550 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccefcd7-7335-4430-9c3a-0b56d0490cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:08.542846 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccefcd7-7335-4430-9c3a-0b56d0490cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:08.543126 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccefcd7-7335-4430-9c3a-0b56d0490cc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:08.544016 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccefcd7-7335-4430-9c3a-0b56d0490cc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:34:08.544327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccefcd7-7335-4430-9c3a-0b56d0490cc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:08.546857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccefcd7-7335-4430-9c3a-0b56d0490cc8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:08.582389 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f604b-d57c-4429-9a7b-f80b805a7cbc None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:08.582780 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 847/1727] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:34:08.596613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f604b-d57c-4429-9a7b-f80b805a7cbc None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:08.618000 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee925808-1dee-4534-821d-3cc7b6f33322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:08.618696 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee925808-1dee-4534-821d-3cc7b6f33322 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:08.619111 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee925808-1dee-4534-821d-3cc7b6f33322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:08.619450 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee925808-1dee-4534-821d-3cc7b6f33322 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:08.621625 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee925808-1dee-4534-821d-3cc7b6f33322 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:08.630730 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee925808-1dee-4534-821d-3cc7b6f33322 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:08.688799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccefcd7-7335-4430-9c3a-0b56d0490cc8 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:08.690703 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1ccefcd7-7335-4430-9c3a-0b56d0490cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:08.730918 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccefcd7-7335-4430-9c3a-0b56d0490cc8 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:08.743100 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce9034-4a11-4cc7-a4b7-8283903c2f66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:08.747450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce9034-4a11-4cc7-a4b7-8283903c2f66 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:08.747739 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce9034-4a11-4cc7-a4b7-8283903c2f66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:08.747981 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce9034-4a11-4cc7-a4b7-8283903c2f66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:08.760038 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-95ce9034-4a11-4cc7-a4b7-8283903c2f66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:08.774969 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 881/1728] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:34:08 2022] POST /identity/v3/projects => generated 365 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:08.776554 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee925808-1dee-4534-821d-3cc7b6f33322 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:08.842833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:08.921859 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee925808-1dee-4534-821d-3cc7b6f33322 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:08.933596 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08fc491-2a6f-4cf1-b687-3a8f4b6f31d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:08.934316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fc491-2a6f-4cf1-b687-3a8f4b6f31d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:08.934607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fc491-2a6f-4cf1-b687-3a8f4b6f31d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:08.934891 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fc491-2a6f-4cf1-b687-3a8f4b6f31d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:08.936505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce9034-4a11-4cc7-a4b7-8283903c2f66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:08.937726 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08fc491-2a6f-4cf1-b687-3a8f4b6f31d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:08.938000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce9034-4a11-4cc7-a4b7-8283903c2f66 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:08.944437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08fc491-2a6f-4cf1-b687-3a8f4b6f31d9 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:08.955422 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 848/1729] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:34:09.029857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 882/1730] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:34:08 2022] POST /identity/v3/users => generated 283 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:09.031870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce9034-4a11-4cc7-a4b7-8283903c2f66 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:09.068350 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce9034-4a11-4cc7-a4b7-8283903c2f66 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:09.086826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08080c60-6f54-4743-a1c0-833374b8a9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:09.087582 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08080c60-6f54-4743-a1c0-833374b8a9d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:09.087982 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08080c60-6f54-4743-a1c0-833374b8a9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:09.088505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08080c60-6f54-4743-a1c0-833374b8a9d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:09.089347 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08080c60-6f54-4743-a1c0-833374b8a9d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:09.091151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08080c60-6f54-4743-a1c0-833374b8a9d1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:09.131633 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08fc491-2a6f-4cf1-b687-3a8f4b6f31d9 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:09.141135 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 849/1731] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:09.155811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08fc491-2a6f-4cf1-b687-3a8f4b6f31d9 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:09.190622 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae9a6f-b740-4d0f-b38a-27c2bb7d7248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:09.192958 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae9a6f-b740-4d0f-b38a-27c2bb7d7248 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:09.193278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae9a6f-b740-4d0f-b38a-27c2bb7d7248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:09.194666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae9a6f-b740-4d0f-b38a-27c2bb7d7248 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:09.205934 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae9a6f-b740-4d0f-b38a-27c2bb7d7248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:09.223526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae9a6f-b740-4d0f-b38a-27c2bb7d7248 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:09.282034 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 883/1732] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:09.283336 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08080c60-6f54-4743-a1c0-833374b8a9d1 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:09.293925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08080c60-6f54-4743-a1c0-833374b8a9d1 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:09.325569 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed7303-b547-4e05-8116-4ce183d48fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:09.326329 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed7303-b547-4e05-8116-4ce183d48fed admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:09.326645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed7303-b547-4e05-8116-4ce183d48fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:09.326941 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed7303-b547-4e05-8116-4ce183d48fed admin admin] PATH_INFO: `/v3/projects/7d5edd8b3e49445187b9052fe50ec13a/users/860e060a11714a6189357ccd3a1caec3/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:09.396422 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed7303-b547-4e05-8116-4ce183d48fed admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d5edd8b3e49445187b9052fe50ec13a, user_id=860e060a11714a6189357ccd3a1caec3, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:09.400348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed7303-b547-4e05-8116-4ce183d48fed admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:09.449923 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 850/1733] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:09 2022] PUT /identity/v3/projects/7d5edd8b3e49445187b9052fe50ec13a/users/860e060a11714a6189357ccd3a1caec3/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:09.456587 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de92365-3ed1-4b22-a5bc-c32e7d816730 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:09.457110 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de92365-3ed1-4b22-a5bc-c32e7d816730 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:09.457201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de92365-3ed1-4b22-a5bc-c32e7d816730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:09.535871 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed7303-b547-4e05-8116-4ce183d48fed admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:09.538239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-07ed7303-b547-4e05-8116-4ce183d48fed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:09.543770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:09.551399 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2de92365-3ed1-4b22-a5bc-c32e7d816730 None None] MFA Rules not processed for user `860e060a11714a6189357ccd3a1caec3`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:09.563567 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2de92365-3ed1-4b22-a5bc-c32e7d816730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:09.881791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed7303-b547-4e05-8116-4ce183d48fed admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:09.904020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3342e479-7f20-4133-9191-a69232c8164b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:09.905376 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342e479-7f20-4133-9191-a69232c8164b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:09.909452 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 884/1734] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:09.922454 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342e479-7f20-4133-9191-a69232c8164b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:09.922739 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342e479-7f20-4133-9191-a69232c8164b admin admin] PATH_INFO: `/v3/users/ccf5a2b5a3f34f7285f437ef2b9a94a1` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:09.924384 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeac1c-2c3e-43d7-8ac5-4a93e73eeec4 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:09.927111 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeac1c-2c3e-43d7-8ac5-4a93e73eeec4 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:09.928063 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3342e479-7f20-4133-9191-a69232c8164b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccf5a2b5a3f34f7285f437ef2b9a94a1)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:09.928721 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeac1c-2c3e-43d7-8ac5-4a93e73eeec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:09.931788 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3342e479-7f20-4133-9191-a69232c8164b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:09.991927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-15aeac1c-2c3e-43d7-8ac5-4a93e73eeec4 None None] MFA Rules not processed for user `4001e8ec57b34f57aae1654fad9217f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:09.993581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-15aeac1c-2c3e-43d7-8ac5-4a93e73eeec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:10.044759 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3342e479-7f20-4133-9191-a69232c8164b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccf5a2b5a3f34f7285f437ef2b9a94a1'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:10.047577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3342e479-7f20-4133-9191-a69232c8164b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccf5a2b5a3f34f7285f437ef2b9a94a1'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:10.061433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3342e479-7f20-4133-9191-a69232c8164b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccf5a2b5a3f34f7285f437ef2b9a94a1'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:10.094338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3342e479-7f20-4133-9191-a69232c8164b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccf5a2b5a3f34f7285f437ef2b9a94a1'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:10.095301 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3342e479-7f20-4133-9191-a69232c8164b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccf5a2b5a3f34f7285f437ef2b9a94a1'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:10.122009 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 851/1735] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:34:09 2022] DELETE /identity/v3/users/ccf5a2b5a3f34f7285f437ef2b9a94a1 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:10.123295 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3342e479-7f20-4133-9191-a69232c8164b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:10.126764 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3342e479-7f20-4133-9191-a69232c8164b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:10.162107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 885/1736] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:10.165229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ecac6-e02a-439d-be44-edfa37fbae2a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:10.169298 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ecac6-e02a-439d-be44-edfa37fbae2a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:10.169829 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ecac6-e02a-439d-be44-edfa37fbae2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:10.211866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2a8ecac6-e02a-439d-be44-edfa37fbae2a None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:10.213163 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2a8ecac6-e02a-439d-be44-edfa37fbae2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:10.235232 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:10.291186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3342e479-7f20-4133-9191-a69232c8164b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:10.310855 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bebf3-d065-4d72-8514-a1f679714ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:10.311949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bebf3-d065-4d72-8514-a1f679714ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:10.312500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bebf3-d065-4d72-8514-a1f679714ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:10.313020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bebf3-d065-4d72-8514-a1f679714ce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:10.313976 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bebf3-d065-4d72-8514-a1f679714ce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:10.316556 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bebf3-d065-4d72-8514-a1f679714ce6 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:10.339035 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 886/1737] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:10.339913 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ecac6-e02a-439d-be44-edfa37fbae2a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:10.341430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2a8ecac6-e02a-439d-be44-edfa37fbae2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:10.386803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 852/1738] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:34:10 2022] POST /identity/v3/projects => generated 363 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:10.388444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bebf3-d065-4d72-8514-a1f679714ce6 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:10.396481 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-058bebf3-d065-4d72-8514-a1f679714ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:10.417129 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:10.574149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bebf3-d065-4d72-8514-a1f679714ce6 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:10.591326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0d406-eeae-4f5d-a01d-2bd939d39839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:10.592320 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0d406-eeae-4f5d-a01d-2bd939d39839 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:10.597238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0d406-eeae-4f5d-a01d-2bd939d39839 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:10.597664 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0d406-eeae-4f5d-a01d-2bd939d39839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:10.629440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ecac6-e02a-439d-be44-edfa37fbae2a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:10.820975 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4489516-6f58-4f9e-b0da-408de4a00e9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:10.821639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4489516-6f58-4f9e-b0da-408de4a00e9f service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:10.821949 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4489516-6f58-4f9e-b0da-408de4a00e9f service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:10.822234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4489516-6f58-4f9e-b0da-408de4a00e9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:10.911975 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0d406-eeae-4f5d-a01d-2bd939d39839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:10.915992 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0d406-eeae-4f5d-a01d-2bd939d39839 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:10.953418 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 853/1739] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:10.967295 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0d406-eeae-4f5d-a01d-2bd939d39839 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:10.993937 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:11.005009 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0d406-eeae-4f5d-a01d-2bd939d39839 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:11.026205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721a31b-2363-4ee0-ab3f-6f5d24de72b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:11.027211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721a31b-2363-4ee0-ab3f-6f5d24de72b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:11.032970 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721a31b-2363-4ee0-ab3f-6f5d24de72b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:11.033563 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721a31b-2363-4ee0-ab3f-6f5d24de72b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:11.037455 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721a31b-2363-4ee0-ab3f-6f5d24de72b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:11.039844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4489516-6f58-4f9e-b0da-408de4a00e9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:11.041500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721a31b-2363-4ee0-ab3f-6f5d24de72b9 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:11.051960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4489516-6f58-4f9e-b0da-408de4a00e9f service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:11.089649 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 887/1740] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 749 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:11.093835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae9f0e-ce41-4691-aec2-167ee210d71a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:11.096155 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae9f0e-ce41-4691-aec2-167ee210d71a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:11.096468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae9f0e-ce41-4691-aec2-167ee210d71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:11.126131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 854/1741] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:34:10 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:11.127418 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721a31b-2363-4ee0-ab3f-6f5d24de72b9 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:11.166023 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721a31b-2363-4ee0-ab3f-6f5d24de72b9 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:11.172908 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-38ae9f0e-ce41-4691-aec2-167ee210d71a None None] MFA Rules not processed for user `7d1b6c388c7642a892c905d16f3b8d50`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:11.178513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-38ae9f0e-ce41-4691-aec2-167ee210d71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:11.187521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2af81-bded-4bd1-9149-563a64a9536a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:11.188185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2af81-bded-4bd1-9149-563a64a9536a admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:11.188458 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2af81-bded-4bd1-9149-563a64a9536a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:11.188738 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2af81-bded-4bd1-9149-563a64a9536a admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:11.195800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2af81-bded-4bd1-9149-563a64a9536a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:11.199636 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2af81-bded-4bd1-9149-563a64a9536a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:11.253970 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 855/1742] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:11.263662 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2af81-bded-4bd1-9149-563a64a9536a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:11.311472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2af81-bded-4bd1-9149-563a64a9536a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:11.336684 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8b8bb-208f-4538-9020-e646e2ff4907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:11.337702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8b8bb-208f-4538-9020-e646e2ff4907 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:11.338245 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8b8bb-208f-4538-9020-e646e2ff4907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:11.339429 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8b8bb-208f-4538-9020-e646e2ff4907 admin admin] PATH_INFO: `/v3/projects/e9d1933937804b3595b8f92a990c1f9b/users/d3a6938502cc4777ab4c7d060dfb1586/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:11.388677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea8b8bb-208f-4538-9020-e646e2ff4907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9d1933937804b3595b8f92a990c1f9b, user_id=d3a6938502cc4777ab4c7d060dfb1586, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:11.391252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea8b8bb-208f-4538-9020-e646e2ff4907 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:11.401980 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 888/1743] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:11.422590 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 856/1744] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:11 2022] PUT /identity/v3/projects/e9d1933937804b3595b8f92a990c1f9b/users/d3a6938502cc4777ab4c7d060dfb1586/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:11.426756 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae9f0e-ce41-4691-aec2-167ee210d71a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:11.446344 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae9f0e-ce41-4691-aec2-167ee210d71a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:11.622684 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c243bc-faf0-46d3-ab92-a23f55fd0de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:11.623385 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c243bc-faf0-46d3-ab92-a23f55fd0de3 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:11.623673 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c243bc-faf0-46d3-ab92-a23f55fd0de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:11.623950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c243bc-faf0-46d3-ab92-a23f55fd0de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:11.624717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c243bc-faf0-46d3-ab92-a23f55fd0de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:11.628457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c243bc-faf0-46d3-ab92-a23f55fd0de3 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:11.660745 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 889/1745] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:11.668263 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8b8bb-208f-4538-9020-e646e2ff4907 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:11.695607 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8b8bb-208f-4538-9020-e646e2ff4907 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:11.704764 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce58fd4-b01c-43cf-8d5b-62fc203258db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:11.705537 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce58fd4-b01c-43cf-8d5b-62fc203258db admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:11.706020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce58fd4-b01c-43cf-8d5b-62fc203258db admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:11.706326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce58fd4-b01c-43cf-8d5b-62fc203258db admin admin] PATH_INFO: `/v3/projects/e9d1933937804b3595b8f92a990c1f9b/users/d3a6938502cc4777ab4c7d060dfb1586/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:11.708542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce58fd4-b01c-43cf-8d5b-62fc203258db admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9d1933937804b3595b8f92a990c1f9b, user_id=d3a6938502cc4777ab4c7d060dfb1586, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:11.713127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce58fd4-b01c-43cf-8d5b-62fc203258db admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:11.755342 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 857/1746] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:11 2022] PUT /identity/v3/projects/e9d1933937804b3595b8f92a990c1f9b/users/d3a6938502cc4777ab4c7d060dfb1586/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:34:11.768366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c479777-4dc7-4962-ae00-a1ca43869f60 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:11.769423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c479777-4dc7-4962-ae00-a1ca43869f60 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:11.769774 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c479777-4dc7-4962-ae00-a1ca43869f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:11.808032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5372f-68f6-492b-9a30-46e9aa1ea63d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:11.808416 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5372f-68f6-492b-9a30-46e9aa1ea63d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:11.808655 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5372f-68f6-492b-9a30-46e9aa1ea63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:11.861265 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-6c479777-4dc7-4962-ae00-a1ca43869f60 None None] MFA Rules not processed for user `d3a6938502cc4777ab4c7d060dfb1586`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:11.866929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6c479777-4dc7-4962-ae00-a1ca43869f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:11.975765 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-9bc5372f-68f6-492b-9a30-46e9aa1ea63d None None] MFA Rules not processed for user `f36c6987ef774a1684c55f885e2a4f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:11.979085 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9bc5372f-68f6-492b-9a30-46e9aa1ea63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:12.037143 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:12.137352 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 890/1747] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:11 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:34:12.138576 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c479777-4dc7-4962-ae00-a1ca43869f60 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:12.140886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6c479777-4dc7-4962-ae00-a1ca43869f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:12.186412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:12.187188 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 858/1748] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Aug 1 15:34:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:12.190016 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5372f-68f6-492b-9a30-46e9aa1ea63d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:12.191868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9bc5372f-68f6-492b-9a30-46e9aa1ea63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:12.297504 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c479777-4dc7-4962-ae00-a1ca43869f60 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:12.310782 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a35472-9108-412a-bd5f-7281115b513f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:12.312005 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a35472-9108-412a-bd5f-7281115b513f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:12.312324 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a35472-9108-412a-bd5f-7281115b513f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:12.312601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a35472-9108-412a-bd5f-7281115b513f admin admin] PATH_INFO: `/v3/projects/9b6b52540e8e4b5a9db60b4bcbc227fc` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:12.315193 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a35472-9108-412a-bd5f-7281115b513f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b6b52540e8e4b5a9db60b4bcbc227fc)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:12.317209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a35472-9108-412a-bd5f-7281115b513f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:12.321632 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5372f-68f6-492b-9a30-46e9aa1ea63d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:12.335002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536717d6-ad94-4c28-b552-c760962f5318 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:12.335667 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536717d6-ad94-4c28-b552-c760962f5318 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:12.336129 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536717d6-ad94-4c28-b552-c760962f5318 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:12.336577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536717d6-ad94-4c28-b552-c760962f5318 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:12.359774 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536717d6-ad94-4c28-b552-c760962f5318 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:12.361305 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536717d6-ad94-4c28-b552-c760962f5318 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:12.392596 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 859/1749] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:12 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:12.406828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-40a35472-9108-412a-bd5f-7281115b513f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b6b52540e8e4b5a9db60b4bcbc227fc', 'request_id': 'req-40a35472-9108-412a-bd5f-7281115b513f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:12.414038 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-40a35472-9108-412a-bd5f-7281115b513f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b6b52540e8e4b5a9db60b4bcbc227fc', 'request_id': 'req-40a35472-9108-412a-bd5f-7281115b513f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:12.424480 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-40a35472-9108-412a-bd5f-7281115b513f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b6b52540e8e4b5a9db60b4bcbc227fc', 'request_id': 'req-40a35472-9108-412a-bd5f-7281115b513f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:12.448121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-40a35472-9108-412a-bd5f-7281115b513f admin admin] The token cache is being invalidate because project 9b6b52540e8e4b5a9db60b4bcbc227fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:12.448470 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-40a35472-9108-412a-bd5f-7281115b513f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:12.449752 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-40a35472-9108-412a-bd5f-7281115b513f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:12.451915 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 891/1750] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:12 2022] DELETE /identity/v3/projects/9b6b52540e8e4b5a9db60b4bcbc227fc => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:12.456058 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536717d6-ad94-4c28-b552-c760962f5318 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:12.472477 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536717d6-ad94-4c28-b552-c760962f5318 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:12.487522 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a35472-9108-412a-bd5f-7281115b513f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:12.490119 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-40a35472-9108-412a-bd5f-7281115b513f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:12.660649 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f09c7b-6e22-4acf-8223-04a4a9949b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:12.662162 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f09c7b-6e22-4acf-8223-04a4a9949b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:12.662563 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f09c7b-6e22-4acf-8223-04a4a9949b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:12.663178 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f09c7b-6e22-4acf-8223-04a4a9949b4f admin admin] PATH_INFO: `/v3/users/936ff0fe3cb3430aab4474478197b0b6` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:12.664779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f09c7b-6e22-4acf-8223-04a4a9949b4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=936ff0fe3cb3430aab4474478197b0b6)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:12.667518 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f09c7b-6e22-4acf-8223-04a4a9949b4f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:12.777529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d2f09c7b-6e22-4acf-8223-04a4a9949b4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '936ff0fe3cb3430aab4474478197b0b6'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:12.778510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d2f09c7b-6e22-4acf-8223-04a4a9949b4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '936ff0fe3cb3430aab4474478197b0b6'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:12.795123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d2f09c7b-6e22-4acf-8223-04a4a9949b4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '936ff0fe3cb3430aab4474478197b0b6'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:12.829210 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:12.844900 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a35472-9108-412a-bd5f-7281115b513f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:12.848021 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d2f09c7b-6e22-4acf-8223-04a4a9949b4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '936ff0fe3cb3430aab4474478197b0b6'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:12.848893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d2f09c7b-6e22-4acf-8223-04a4a9949b4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '936ff0fe3cb3430aab4474478197b0b6'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:12.861018 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623e097-3482-4966-8a2e-d0b5991e1b16 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6873a3e440bd4b63b0082fc85beef49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:12.861681 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623e097-3482-4966-8a2e-d0b5991e1b16 service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:12.862008 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623e097-3482-4966-8a2e-d0b5991e1b16 service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:12.862287 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623e097-3482-4966-8a2e-d0b5991e1b16 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:12.867368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2623e097-3482-4966-8a2e-d0b5991e1b16 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:12.904328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 860/1751] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:34:12 2022] DELETE /identity/v3/users/936ff0fe3cb3430aab4474478197b0b6 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:34:12.906343 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f09c7b-6e22-4acf-8223-04a4a9949b4f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:12.921507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d2f09c7b-6e22-4acf-8223-04a4a9949b4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:12.941029 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:13.048787 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2623e097-3482-4966-8a2e-d0b5991e1b16 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:13.050723 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2623e097-3482-4966-8a2e-d0b5991e1b16 service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:13.079489 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 892/1752] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Aug 1 15:34:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:34:13.084366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d2f14-37e9-41fa-98e5-d2bab2aafbd1 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:13.084782 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d2f14-37e9-41fa-98e5-d2bab2aafbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:13.085169 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d2f14-37e9-41fa-98e5-d2bab2aafbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:13.101740 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f09c7b-6e22-4acf-8223-04a4a9949b4f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:13.106713 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276bfe6-db26-4558-8cd4-bc4684ec2d85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:13.107372 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276bfe6-db26-4558-8cd4-bc4684ec2d85 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:13.107675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276bfe6-db26-4558-8cd4-bc4684ec2d85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:13.107957 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276bfe6-db26-4558-8cd4-bc4684ec2d85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:13.109935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0276bfe6-db26-4558-8cd4-bc4684ec2d85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:13.130325 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-209d2f14-37e9-41fa-98e5-d2bab2aafbd1 None None] MFA Rules not processed for user `79cefb8ac83f470e86d2f28845854d18`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:13.133242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-209d2f14-37e9-41fa-98e5-d2bab2aafbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:13.244586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276bfe6-db26-4558-8cd4-bc4684ec2d85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:13.249809 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276bfe6-db26-4558-8cd4-bc4684ec2d85 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:13.270043 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 861/1753] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:13.431748 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 893/1754] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:13.433843 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276bfe6-db26-4558-8cd4-bc4684ec2d85 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:13.436991 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0276bfe6-db26-4558-8cd4-bc4684ec2d85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:13.695751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276bfe6-db26-4558-8cd4-bc4684ec2d85 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:13.728670 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a17947-2f2e-467c-a067-2b1209ddb121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:13.733612 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a17947-2f2e-467c-a067-2b1209ddb121 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:13.734667 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a17947-2f2e-467c-a067-2b1209ddb121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:13.735698 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a17947-2f2e-467c-a067-2b1209ddb121 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:13.737194 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a17947-2f2e-467c-a067-2b1209ddb121 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:13.752358 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a17947-2f2e-467c-a067-2b1209ddb121 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:13.823813 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d2f14-37e9-41fa-98e5-d2bab2aafbd1 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:13.824112 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 862/1755] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:34:13 2022] POST /identity/v3/projects => generated 405 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:13.850282 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d2f14-37e9-41fa-98e5-d2bab2aafbd1 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:14.118524 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d6365-1e77-4055-808c-74039a499491 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:14.119547 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d6365-1e77-4055-808c-74039a499491 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:14.122022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d6365-1e77-4055-808c-74039a499491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:14.135941 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df675f9-b460-4f0e-b794-3e5d0f769ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:14.144172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df675f9-b460-4f0e-b794-3e5d0f769ead admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:14.144572 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df675f9-b460-4f0e-b794-3e5d0f769ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:14.150316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df675f9-b460-4f0e-b794-3e5d0f769ead admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:14.151200 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df675f9-b460-4f0e-b794-3e5d0f769ead admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:14.154093 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df675f9-b460-4f0e-b794-3e5d0f769ead admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:14.204186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 894/1756] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:34:13 2022] POST /identity/v3/users => generated 309 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:34:14.205354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df675f9-b460-4f0e-b794-3e5d0f769ead admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:14.230892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-b55d6365-1e77-4055-808c-74039a499491 None None] MFA Rules not processed for user `5f0b12ba782244d7b44783c3f3532bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:14.238897 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b55d6365-1e77-4055-808c-74039a499491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:14.245860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df675f9-b460-4f0e-b794-3e5d0f769ead admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:14.267810 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8f672-622b-43b1-bbb6-b2078c535432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:14.268629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8f672-622b-43b1-bbb6-b2078c535432 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:14.269240 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8f672-622b-43b1-bbb6-b2078c535432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:14.269364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8f672-622b-43b1-bbb6-b2078c535432 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:14.278317 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a8f672-622b-43b1-bbb6-b2078c535432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:14.278554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:14.280931 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a8f672-622b-43b1-bbb6-b2078c535432 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:14.305495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 895/1757] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:14.309655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8f672-622b-43b1-bbb6-b2078c535432 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:14.358544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8f672-622b-43b1-bbb6-b2078c535432 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:14.380550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6f42f-029a-4bda-ba8a-656e98f7dcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:14.381772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6f42f-029a-4bda-ba8a-656e98f7dcee admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:14.382419 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6f42f-029a-4bda-ba8a-656e98f7dcee admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:14.383050 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6f42f-029a-4bda-ba8a-656e98f7dcee admin admin] PATH_INFO: `/v3/projects/28e4231ebaf344a9928809d01227ead1/users/55996ae2c02840d698b422f0dc9ad22d/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:14.435446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6f42f-029a-4bda-ba8a-656e98f7dcee admin admin] RBAC: Authorizing `identity:create_grant(project_id=28e4231ebaf344a9928809d01227ead1, user_id=55996ae2c02840d698b422f0dc9ad22d, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:14.441660 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6f42f-029a-4bda-ba8a-656e98f7dcee admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:14.472840 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 863/1758] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:14.487632 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ef615-f2bb-4957-94b0-3c8c29a62384 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:14.488120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ef615-f2bb-4957-94b0-3c8c29a62384 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:14.488954 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ef615-f2bb-4957-94b0-3c8c29a62384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:14.496797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 896/1759] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:14 2022] PUT /identity/v3/projects/28e4231ebaf344a9928809d01227ead1/users/55996ae2c02840d698b422f0dc9ad22d/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:14.498314 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6f42f-029a-4bda-ba8a-656e98f7dcee admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:14.519407 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:14.547874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-805ef615-f2bb-4957-94b0-3c8c29a62384 None None] MFA Rules not processed for user `5f0b12ba782244d7b44783c3f3532bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:14.556187 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-805ef615-f2bb-4957-94b0-3c8c29a62384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:14.591973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6f42f-029a-4bda-ba8a-656e98f7dcee admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:14.601046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901f2c7-0e43-4dcb-bbc6-cf79ef47872c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:14.602910 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901f2c7-0e43-4dcb-bbc6-cf79ef47872c admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:14.603846 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901f2c7-0e43-4dcb-bbc6-cf79ef47872c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:14.604378 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901f2c7-0e43-4dcb-bbc6-cf79ef47872c admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:14.605550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901f2c7-0e43-4dcb-bbc6-cf79ef47872c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:14.627349 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901f2c7-0e43-4dcb-bbc6-cf79ef47872c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:14.652537 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 897/1760] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:14.655178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901f2c7-0e43-4dcb-bbc6-cf79ef47872c admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:14.667893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901f2c7-0e43-4dcb-bbc6-cf79ef47872c admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:14.675158 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadae37-fa0e-41c1-8c0d-43f4ca991e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:14.676313 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadae37-fa0e-41c1-8c0d-43f4ca991e71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:14.676818 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadae37-fa0e-41c1-8c0d-43f4ca991e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:14.677308 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadae37-fa0e-41c1-8c0d-43f4ca991e71 admin admin] PATH_INFO: `/v3/projects/28e4231ebaf344a9928809d01227ead1/users/55996ae2c02840d698b422f0dc9ad22d/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:14.680007 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadae37-fa0e-41c1-8c0d-43f4ca991e71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28e4231ebaf344a9928809d01227ead1, user_id=55996ae2c02840d698b422f0dc9ad22d, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:14.683363 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadae37-fa0e-41c1-8c0d-43f4ca991e71 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:14.698307 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 898/1761] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:14 2022] PUT /identity/v3/projects/28e4231ebaf344a9928809d01227ead1/users/55996ae2c02840d698b422f0dc9ad22d/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:14.803129 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 864/1762] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:15.877200 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadae37-fa0e-41c1-8c0d-43f4ca991e71 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:15.879751 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-daadae37-fa0e-41c1-8c0d-43f4ca991e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:16.176655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadae37-fa0e-41c1-8c0d-43f4ca991e71 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:16.221059 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afba036-0f6f-48ef-afcc-62b334e7f3e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:16.221774 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afba036-0f6f-48ef-afcc-62b334e7f3e0 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:16.222272 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afba036-0f6f-48ef-afcc-62b334e7f3e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:16.222801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afba036-0f6f-48ef-afcc-62b334e7f3e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:16.230704 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe25ae-bd97-4031-a81e-523990aa63ea None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:16.233546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe25ae-bd97-4031-a81e-523990aa63ea None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:16.234622 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe25ae-bd97-4031-a81e-523990aa63ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:16.344259 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-8cbe25ae-bd97-4031-a81e-523990aa63ea None None] MFA Rules not processed for user `76311c7173b442c0af2c745e3e70a07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:16.352064 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8cbe25ae-bd97-4031-a81e-523990aa63ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:16.496631 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afba036-0f6f-48ef-afcc-62b334e7f3e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:16.502350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afba036-0f6f-48ef-afcc-62b334e7f3e0 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:16.547085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 899/1763] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 670 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:16.609575 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 865/1764] 10.222.0.46 () {58 vars in 1015 bytes} [Mon Aug 1 15:34:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:16.998235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afba036-0f6f-48ef-afcc-62b334e7f3e0 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:17.000452 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8afba036-0f6f-48ef-afcc-62b334e7f3e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:17.029119 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afba036-0f6f-48ef-afcc-62b334e7f3e0 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:17.048173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7fb776-d181-4777-80a6-6b030eae5550 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:17.049306 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fb776-d181-4777-80a6-6b030eae5550 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:17.049677 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fb776-d181-4777-80a6-6b030eae5550 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:17.051096 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fb776-d181-4777-80a6-6b030eae5550 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:17.067532 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7fb776-d181-4777-80a6-6b030eae5550 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:17.069409 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7fb776-d181-4777-80a6-6b030eae5550 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:17.104206 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 900/1765] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:19.328593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe25ae-bd97-4031-a81e-523990aa63ea None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:19.330762 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8cbe25ae-bd97-4031-a81e-523990aa63ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:19.474851 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6167bdf-10c4-43fd-b501-9ebc53df5a34 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:19.476263 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6167bdf-10c4-43fd-b501-9ebc53df5a34 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:19.476955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6167bdf-10c4-43fd-b501-9ebc53df5a34 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:19.483360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 901/1766] 10.222.0.83 () {54 vars in 912 bytes} [Mon Aug 1 15:34:19 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:34:19.490126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17c31e-4db5-4df0-ba2d-a1b6b7dd9fa2 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:19.491125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17c31e-4db5-4df0-ba2d-a1b6b7dd9fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:19.491740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17c31e-4db5-4df0-ba2d-a1b6b7dd9fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:19.522551 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe25ae-bd97-4031-a81e-523990aa63ea None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:19.532128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ff116-ac2b-4d97-be45-329346b5d7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:19.533079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ff116-ac2b-4d97-be45-329346b5d7de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:19.533373 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ff116-ac2b-4d97-be45-329346b5d7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:19.533659 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ff116-ac2b-4d97-be45-329346b5d7de admin admin] PATH_INFO: `/v3/projects/4789ef07a02a4cf2a77339ebaf0b254d` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:19.536009 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ff116-ac2b-4d97-be45-329346b5d7de admin admin] RBAC: Authorizing `identity:delete_project(project_id=4789ef07a02a4cf2a77339ebaf0b254d)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:19.536815 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-6f17c31e-4db5-4df0-ba2d-a1b6b7dd9fa2 None None] MFA Rules not processed for user `76311c7173b442c0af2c745e3e70a07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:19.538045 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ff116-ac2b-4d97-be45-329346b5d7de admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:19.540470 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6f17c31e-4db5-4df0-ba2d-a1b6b7dd9fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:19.564154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 902/1767] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Aug 1 15:34:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:34:19.605719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:19.752777 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-325ff116-ac2b-4d97-be45-329346b5d7de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4789ef07a02a4cf2a77339ebaf0b254d', 'request_id': 'req-325ff116-ac2b-4d97-be45-329346b5d7de'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:19.818587 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-325ff116-ac2b-4d97-be45-329346b5d7de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4789ef07a02a4cf2a77339ebaf0b254d', 'request_id': 'req-325ff116-ac2b-4d97-be45-329346b5d7de'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:19.819586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17c31e-4db5-4df0-ba2d-a1b6b7dd9fa2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:19.836434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17c31e-4db5-4df0-ba2d-a1b6b7dd9fa2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:19.860543 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:19.869550 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-325ff116-ac2b-4d97-be45-329346b5d7de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4789ef07a02a4cf2a77339ebaf0b254d', 'request_id': 'req-325ff116-ac2b-4d97-be45-329346b5d7de'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:19.930398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-325ff116-ac2b-4d97-be45-329346b5d7de admin admin] The token cache is being invalidate because project 4789ef07a02a4cf2a77339ebaf0b254d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:19.930876 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-325ff116-ac2b-4d97-be45-329346b5d7de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:19.934646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-325ff116-ac2b-4d97-be45-329346b5d7de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:19.943872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 866/1768] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:19 2022] DELETE /identity/v3/projects/4789ef07a02a4cf2a77339ebaf0b254d => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:19.945780 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d8994-365c-4362-a811-acf77d5134a9 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:19.948404 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d8994-365c-4362-a811-acf77d5134a9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:19.949931 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d8994-365c-4362-a811-acf77d5134a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:20.025016 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-2f1d8994-365c-4362-a811-acf77d5134a9 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:20.034635 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2f1d8994-365c-4362-a811-acf77d5134a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:20.042784 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc85e47-3b9c-4017-8886-e3a8b206c4c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:20.046576 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc85e47-3b9c-4017-8886-e3a8b206c4c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:20.048309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc85e47-3b9c-4017-8886-e3a8b206c4c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:20.055566 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc85e47-3b9c-4017-8886-e3a8b206c4c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:20.067140 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-dfc85e47-3b9c-4017-8886-e3a8b206c4c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:20.104060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc85e47-3b9c-4017-8886-e3a8b206c4c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:20.108221 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc85e47-3b9c-4017-8886-e3a8b206c4c2 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:20.183812 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 903/1769] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:34:20.286759 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 867/1770] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:34:20.289409 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e17b7c-e8e8-4e1b-8c4c-34c41917fff0 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:20.290462 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e17b7c-e8e8-4e1b-8c4c-34c41917fff0 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:20.291211 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e17b7c-e8e8-4e1b-8c4c-34c41917fff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:20.398823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-03e17b7c-e8e8-4e1b-8c4c-34c41917fff0 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:20.405916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-03e17b7c-e8e8-4e1b-8c4c-34c41917fff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:20.452228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 904/1771] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:20.453714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d8994-365c-4362-a811-acf77d5134a9 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:20.475256 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d8994-365c-4362-a811-acf77d5134a9 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:20.483071 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba09aa-2cb9-4734-b6c5-5c943051ad06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:20.484050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba09aa-2cb9-4734-b6c5-5c943051ad06 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:20.484459 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba09aa-2cb9-4734-b6c5-5c943051ad06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:20.484978 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba09aa-2cb9-4734-b6c5-5c943051ad06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:20.485997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba09aa-2cb9-4734-b6c5-5c943051ad06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:34:20.486736 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba09aa-2cb9-4734-b6c5-5c943051ad06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:20.489878 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba09aa-2cb9-4734-b6c5-5c943051ad06 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:20.516686 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 868/1772] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:34:20.518644 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e17b7c-e8e8-4e1b-8c4c-34c41917fff0 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:20.533552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e17b7c-e8e8-4e1b-8c4c-34c41917fff0 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:20.541090 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e685a37-1fc0-4718-9ed4-f3db4c86ae3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:20.542530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e685a37-1fc0-4718-9ed4-f3db4c86ae3f admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:20.543251 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e685a37-1fc0-4718-9ed4-f3db4c86ae3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:20.544032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e685a37-1fc0-4718-9ed4-f3db4c86ae3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:20.545493 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e685a37-1fc0-4718-9ed4-f3db4c86ae3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:20.585786 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e685a37-1fc0-4718-9ed4-f3db4c86ae3f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:20.666824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 905/1773] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:34:20 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:20.668163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba09aa-2cb9-4734-b6c5-5c943051ad06 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:20.700982 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba09aa-2cb9-4734-b6c5-5c943051ad06 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:20.743825 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0f3b2-5e04-43cc-83ee-79396193329e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:20.744336 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0f3b2-5e04-43cc-83ee-79396193329e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:20.745749 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0f3b2-5e04-43cc-83ee-79396193329e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:20.825607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-b4c0f3b2-5e04-43cc-83ee-79396193329e None None] MFA Rules not processed for user `55996ae2c02840d698b422f0dc9ad22d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:20.828332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b4c0f3b2-5e04-43cc-83ee-79396193329e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:20.917375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48308e55-1738-45e7-80fd-6611dbaf7029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:20.918392 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48308e55-1738-45e7-80fd-6611dbaf7029 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:20.918887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48308e55-1738-45e7-80fd-6611dbaf7029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:20.919368 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48308e55-1738-45e7-80fd-6611dbaf7029 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:20.920834 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48308e55-1738-45e7-80fd-6611dbaf7029 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:20.925994 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48308e55-1738-45e7-80fd-6611dbaf7029 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:20.943770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 906/1774] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:20 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:34:20.951830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7eb104-ffab-43fa-9b44-ab3a9698c6cb None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:20.952818 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7eb104-ffab-43fa-9b44-ab3a9698c6cb None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:20.953224 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7eb104-ffab-43fa-9b44-ab3a9698c6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:20.973536 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 869/1775] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:34:20 2022] POST /identity/v3/users => generated 287 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:20.978595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d86d71-486f-4398-8e1f-3b5013439780 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:20.979099 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d86d71-486f-4398-8e1f-3b5013439780 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:20.979670 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d86d71-486f-4398-8e1f-3b5013439780 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:20.981980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 870/1776] 10.222.0.83 () {54 vars in 921 bytes} [Mon Aug 1 15:34:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:34:20.983027 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d86d71-486f-4398-8e1f-3b5013439780 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:20.994716 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-7d7eb104-ffab-43fa-9b44-ab3a9698c6cb None None] MFA Rules not processed for user `4001e8ec57b34f57aae1654fad9217f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:20.997819 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7d7eb104-ffab-43fa-9b44-ab3a9698c6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:21.004608 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d86d71-486f-4398-8e1f-3b5013439780 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:21.009670 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b436fd6-3728-4fd8-b4d7-3f4a32135bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:21.010618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b436fd6-3728-4fd8-b4d7-3f4a32135bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:21.010958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b436fd6-3728-4fd8-b4d7-3f4a32135bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:21.011331 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b436fd6-3728-4fd8-b4d7-3f4a32135bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:21.012453 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b436fd6-3728-4fd8-b4d7-3f4a32135bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:21.014415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b436fd6-3728-4fd8-b4d7-3f4a32135bd9 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:21.038090 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 871/1777] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:21.043363 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075723e-47ce-4034-8c76-67d9239c7371 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:21.044768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075723e-47ce-4034-8c76-67d9239c7371 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:21.047120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075723e-47ce-4034-8c76-67d9239c7371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:21.062341 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:21.079745 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a075723e-47ce-4034-8c76-67d9239c7371 None None] MFA Rules not processed for user `25dc41ffc41c47fdbeec06110d04610c`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:21.081659 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a075723e-47ce-4034-8c76-67d9239c7371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:21.149373 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 907/1778] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:21.151073 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7eb104-ffab-43fa-9b44-ab3a9698c6cb None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:21.174452 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7eb104-ffab-43fa-9b44-ab3a9698c6cb None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:21.193208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 872/1779] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Aug 1 15:34:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:21.196751 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377ad93-441f-447e-a85f-1250eb446f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:21.200189 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377ad93-441f-447e-a85f-1250eb446f87 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:21.200703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377ad93-441f-447e-a85f-1250eb446f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:21.201140 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377ad93-441f-447e-a85f-1250eb446f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:21.202696 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377ad93-441f-447e-a85f-1250eb446f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:21.207123 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377ad93-441f-447e-a85f-1250eb446f87 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:21.234364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 908/1780] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:21.240226 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075723e-47ce-4034-8c76-67d9239c7371 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:21.257941 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075723e-47ce-4034-8c76-67d9239c7371 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:21.264607 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51e9bb-a5b3-4ee2-b603-8dc339578b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:21.265308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51e9bb-a5b3-4ee2-b603-8dc339578b44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:21.265692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51e9bb-a5b3-4ee2-b603-8dc339578b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:21.266075 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51e9bb-a5b3-4ee2-b603-8dc339578b44 admin admin] PATH_INFO: `/v3/projects/43b5a672819a44f5bb5597a2d407bcda/users/e7e5c59af98b45dab5fff8fdc83d6cb2/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:21.290501 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51e9bb-a5b3-4ee2-b603-8dc339578b44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43b5a672819a44f5bb5597a2d407bcda, user_id=e7e5c59af98b45dab5fff8fdc83d6cb2, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:21.294191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51e9bb-a5b3-4ee2-b603-8dc339578b44 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:21.320065 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 873/1781] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:21 2022] PUT /identity/v3/projects/43b5a672819a44f5bb5597a2d407bcda/users/e7e5c59af98b45dab5fff8fdc83d6cb2/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:21.325797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483564f-b41c-4348-8550-27d478e61395 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:21.327656 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483564f-b41c-4348-8550-27d478e61395 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:21.328128 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483564f-b41c-4348-8550-27d478e61395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:21.338107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:21.365933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-3483564f-b41c-4348-8550-27d478e61395 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:21.365933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3483564f-b41c-4348-8550-27d478e61395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:21.549402 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 909/1782] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:21.574340 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51e9bb-a5b3-4ee2-b603-8dc339578b44 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:21.577107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6c51e9bb-a5b3-4ee2-b603-8dc339578b44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:21.771028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51e9bb-a5b3-4ee2-b603-8dc339578b44 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:21.778463 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0c937-c6ad-4c74-a9b1-376c732e7ba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:21.779455 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0c937-c6ad-4c74-a9b1-376c732e7ba1 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:21.780068 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0c937-c6ad-4c74-a9b1-376c732e7ba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:21.780513 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0c937-c6ad-4c74-a9b1-376c732e7ba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:21.825430 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0c937-c6ad-4c74-a9b1-376c732e7ba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:21.831086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0c937-c6ad-4c74-a9b1-376c732e7ba1 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:21.882930 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483564f-b41c-4348-8550-27d478e61395 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:21.884696 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3483564f-b41c-4348-8550-27d478e61395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:21.933596 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 874/1783] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:22.408065 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483564f-b41c-4348-8550-27d478e61395 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:22.433564 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4459ee-8996-41d8-aff1-57a6b6442ee6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:22.434613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4459ee-8996-41d8-aff1-57a6b6442ee6 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:22.435156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4459ee-8996-41d8-aff1-57a6b6442ee6 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:22.436276 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4459ee-8996-41d8-aff1-57a6b6442ee6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:22.454890 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4459ee-8996-41d8-aff1-57a6b6442ee6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:22.458715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4459ee-8996-41d8-aff1-57a6b6442ee6 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:22.524323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 910/1784] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:34:24.102144 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e787e-dd55-4338-a718-e4ec4f94bcb8 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:24.102742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e787e-dd55-4338-a718-e4ec4f94bcb8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:24.102742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e787e-dd55-4338-a718-e4ec4f94bcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:24.155620 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d36e787e-dd55-4338-a718-e4ec4f94bcb8 None None] MFA Rules not processed for user `5f0b12ba782244d7b44783c3f3532bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:24.157118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d36e787e-dd55-4338-a718-e4ec4f94bcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:24.233696 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3674f4-8f89-4311-bf4f-7bc9de431de6 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:24.234880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3674f4-8f89-4311-bf4f-7bc9de431de6 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:24.235357 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3674f4-8f89-4311-bf4f-7bc9de431de6 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:24.237283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 911/1785] 10.222.0.83 () {54 vars in 921 bytes} [Mon Aug 1 15:34:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:34:24.314245 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60e774-e2b4-4bee-992c-faea498873ad None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:24.315073 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60e774-e2b4-4bee-992c-faea498873ad None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:24.315929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60e774-e2b4-4bee-992c-faea498873ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:24.341586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 875/1786] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:24.382530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-ca60e774-e2b4-4bee-992c-faea498873ad None None] MFA Rules not processed for user `ef57a9394b114eb3ab499133b5d40781`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:24.384634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ca60e774-e2b4-4bee-992c-faea498873ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:24.504389 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 912/1787] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Aug 1 15:34:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:34:24.820016 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e787e-dd55-4338-a718-e4ec4f94bcb8 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:24.822829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d36e787e-dd55-4338-a718-e4ec4f94bcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:24.932213 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e787e-dd55-4338-a718-e4ec4f94bcb8 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:24.975976 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901976c-5c39-4d76-9b0a-f10a8a74daaa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:24.980960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901976c-5c39-4d76-9b0a-f10a8a74daaa service placement] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:24.981725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901976c-5c39-4d76-9b0a-f10a8a74daaa service placement] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:24.983697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901976c-5c39-4d76-9b0a-f10a8a74daaa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:25.038141 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901976c-5c39-4d76-9b0a-f10a8a74daaa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:25.043665 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901976c-5c39-4d76-9b0a-f10a8a74daaa service placement] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:25.131313 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 876/1788] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:27.281398 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca60e774-e2b4-4bee-992c-faea498873ad None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:27.283409 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ca60e774-e2b4-4bee-992c-faea498873ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:27.321439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8576870-314b-48fa-b215-38dc82e08df9 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:27.322034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8576870-314b-48fa-b215-38dc82e08df9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:27.322270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8576870-314b-48fa-b215-38dc82e08df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:27.328346 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca60e774-e2b4-4bee-992c-faea498873ad None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:27.353433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:27.354317 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c51a2-eaec-4436-8090-13d345b80e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:27.355255 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c51a2-eaec-4436-8090-13d345b80e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:27.355772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c51a2-eaec-4436-8090-13d345b80e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:27.356203 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c51a2-eaec-4436-8090-13d345b80e1f admin admin] PATH_INFO: `/v3/users/860e060a11714a6189357ccd3a1caec3` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:27.358082 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c51a2-eaec-4436-8090-13d345b80e1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=860e060a11714a6189357ccd3a1caec3)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:27.361548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c51a2-eaec-4436-8090-13d345b80e1f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:27.423891 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f8576870-314b-48fa-b215-38dc82e08df9 None None] MFA Rules not processed for user `e7e5c59af98b45dab5fff8fdc83d6cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:27.426363 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f8576870-314b-48fa-b215-38dc82e08df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:27.464439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:27.513220 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-467c51a2-eaec-4436-8090-13d345b80e1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '860e060a11714a6189357ccd3a1caec3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:27.516855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-467c51a2-eaec-4436-8090-13d345b80e1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '860e060a11714a6189357ccd3a1caec3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:27.544320 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-467c51a2-eaec-4436-8090-13d345b80e1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '860e060a11714a6189357ccd3a1caec3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:27.587117 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-467c51a2-eaec-4436-8090-13d345b80e1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '860e060a11714a6189357ccd3a1caec3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:27.611154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-467c51a2-eaec-4436-8090-13d345b80e1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '860e060a11714a6189357ccd3a1caec3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:27.629121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 913/1789] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:34:27 2022] DELETE /identity/v3/users/860e060a11714a6189357ccd3a1caec3 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:34:27.635261 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6153c8f-5daf-4bec-97e4-599346d831ce None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:27.635772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6153c8f-5daf-4bec-97e4-599346d831ce None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:27.636286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6153c8f-5daf-4bec-97e4-599346d831ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:27.704380 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 877/1790] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:27.710743 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8576870-314b-48fa-b215-38dc82e08df9 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:27.713865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f8576870-314b-48fa-b215-38dc82e08df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:27.724397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e6153c8f-5daf-4bec-97e4-599346d831ce None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:27.726059 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e6153c8f-5daf-4bec-97e4-599346d831ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:27.739010 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8576870-314b-48fa-b215-38dc82e08df9 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:27.894169 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c6bbb-c9dd-4e7d-b52a-46bb0e947fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:27.898742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c6bbb-c9dd-4e7d-b52a-46bb0e947fef admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:27.898742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c6bbb-c9dd-4e7d-b52a-46bb0e947fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:27.898742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c6bbb-c9dd-4e7d-b52a-46bb0e947fef admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:27.898742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c6bbb-c9dd-4e7d-b52a-46bb0e947fef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:27.914055 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c6bbb-c9dd-4e7d-b52a-46bb0e947fef admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:27.984957 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 878/1791] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:34:27 2022] POST /identity/v3/projects => generated 373 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:27.991257 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c6bbb-c9dd-4e7d-b52a-46bb0e947fef admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:28.016470 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 914/1792] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:28.070236 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6153c8f-5daf-4bec-97e4-599346d831ce None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:28.070236 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e6153c8f-5daf-4bec-97e4-599346d831ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:28.263589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c6bbb-c9dd-4e7d-b52a-46bb0e947fef admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:28.282312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7149270-6684-4404-a773-464d26a9dd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:28.283231 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6153c8f-5daf-4bec-97e4-599346d831ce None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:28.284263 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7149270-6684-4404-a773-464d26a9dd0b admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:28.284588 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7149270-6684-4404-a773-464d26a9dd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:28.284875 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7149270-6684-4404-a773-464d26a9dd0b admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:28.285593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7149270-6684-4404-a773-464d26a9dd0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:28.290025 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7149270-6684-4404-a773-464d26a9dd0b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:28.378729 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 879/1793] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:34:27 2022] POST /identity/v3/users => generated 293 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:28.390871 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7149270-6684-4404-a773-464d26a9dd0b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:28.423521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7149270-6684-4404-a773-464d26a9dd0b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:28.435252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f3af0-56d6-4d99-b298-56e1ecfd62f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:28.436381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b35576-070f-4b1b-975a-7e405daa64f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:28.437583 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b35576-070f-4b1b-975a-7e405daa64f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:28.438286 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f3af0-56d6-4d99-b298-56e1ecfd62f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:28.438961 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b35576-070f-4b1b-975a-7e405daa64f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:28.440215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f3af0-56d6-4d99-b298-56e1ecfd62f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:28.440715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b35576-070f-4b1b-975a-7e405daa64f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:28.441250 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f3af0-56d6-4d99-b298-56e1ecfd62f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:28.456923 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f3af0-56d6-4d99-b298-56e1ecfd62f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:28.463035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f3af0-56d6-4d99-b298-56e1ecfd62f8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:28.481011 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:28.486232 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b35576-070f-4b1b-975a-7e405daa64f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:28.502793 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b35576-070f-4b1b-975a-7e405daa64f0 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:28.512127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 880/1794] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:28.514049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f3af0-56d6-4d99-b298-56e1ecfd62f8 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:28.514874 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:28.539440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 915/1795] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:28.544510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f3af0-56d6-4d99-b298-56e1ecfd62f8 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:28.567836 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85985873-77e6-430e-986b-0e9bf9159e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:28.572513 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85985873-77e6-430e-986b-0e9bf9159e19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:28.574340 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85985873-77e6-430e-986b-0e9bf9159e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:28.574531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85985873-77e6-430e-986b-0e9bf9159e19 admin admin] PATH_INFO: `/v3/projects/cded9e11aba44bbd9c7140de87e0ec63/users/8daf22a6b42844b3a8db7affe1016a84/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:28.622160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85985873-77e6-430e-986b-0e9bf9159e19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cded9e11aba44bbd9c7140de87e0ec63, user_id=8daf22a6b42844b3a8db7affe1016a84, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:28.625296 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85985873-77e6-430e-986b-0e9bf9159e19 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:28.649092 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704e535-41a6-44bc-ac7f-9abd0360ac2b None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:28.650193 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704e535-41a6-44bc-ac7f-9abd0360ac2b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:28.650987 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704e535-41a6-44bc-ac7f-9abd0360ac2b None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:28.656861 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 916/1796] 10.222.0.83 () {54 vars in 921 bytes} [Mon Aug 1 15:34:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:34:28.684118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10a533-bc75-47d7-bd41-a6cfa5bb3359 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:28.685339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10a533-bc75-47d7-bd41-a6cfa5bb3359 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:28.686721 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10a533-bc75-47d7-bd41-a6cfa5bb3359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:28.700012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 881/1797] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:28 2022] PUT /identity/v3/projects/cded9e11aba44bbd9c7140de87e0ec63/users/8daf22a6b42844b3a8db7affe1016a84/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:28.708561 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85985873-77e6-430e-986b-0e9bf9159e19 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:28.759708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-ef10a533-bc75-47d7-bd41-a6cfa5bb3359 None None] MFA Rules not processed for user `25dc41ffc41c47fdbeec06110d04610c`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:28.767207 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ef10a533-bc75-47d7-bd41-a6cfa5bb3359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:28.909192 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85985873-77e6-430e-986b-0e9bf9159e19 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:28.916999 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec98992-524e-4b03-afd9-328d5840bc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:28.917805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec98992-524e-4b03-afd9-328d5840bc1e admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:28.918385 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec98992-524e-4b03-afd9-328d5840bc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:28.918545 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec98992-524e-4b03-afd9-328d5840bc1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:28.919735 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec98992-524e-4b03-afd9-328d5840bc1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:28.923154 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec98992-524e-4b03-afd9-328d5840bc1e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:28.955264 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 882/1798] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:28.960891 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 917/1799] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Aug 1 15:34:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:28.962730 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10a533-bc75-47d7-bd41-a6cfa5bb3359 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:28.996022 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10a533-bc75-47d7-bd41-a6cfa5bb3359 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:29.003868 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042837c-70c3-4768-bc9c-b53e0ae6926b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:29.004756 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042837c-70c3-4768-bc9c-b53e0ae6926b admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:29.005052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042837c-70c3-4768-bc9c-b53e0ae6926b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:29.005338 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042837c-70c3-4768-bc9c-b53e0ae6926b admin admin] PATH_INFO: `/v3/projects/cded9e11aba44bbd9c7140de87e0ec63/users/8daf22a6b42844b3a8db7affe1016a84/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:29.007584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042837c-70c3-4768-bc9c-b53e0ae6926b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cded9e11aba44bbd9c7140de87e0ec63, user_id=8daf22a6b42844b3a8db7affe1016a84, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:29.010903 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042837c-70c3-4768-bc9c-b53e0ae6926b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:29.028860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 918/1800] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:28 2022] PUT /identity/v3/projects/cded9e11aba44bbd9c7140de87e0ec63/users/8daf22a6b42844b3a8db7affe1016a84/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:30.471428 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec98992-524e-4b03-afd9-328d5840bc1e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:30.475327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-eec98992-524e-4b03-afd9-328d5840bc1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:30.521227 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec98992-524e-4b03-afd9-328d5840bc1e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:30.674936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042837c-70c3-4768-bc9c-b53e0ae6926b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:30.677631 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1042837c-70c3-4768-bc9c-b53e0ae6926b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:30.679398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408b4a9-f9d1-465c-af41-cc2be818bc69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:30.679637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408b4a9-f9d1-465c-af41-cc2be818bc69 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:30.680054 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408b4a9-f9d1-465c-af41-cc2be818bc69 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:30.681638 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408b4a9-f9d1-465c-af41-cc2be818bc69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:30.696971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9408b4a9-f9d1-465c-af41-cc2be818bc69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:30.703349 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9408b4a9-f9d1-465c-af41-cc2be818bc69 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:30.747132 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 883/1801] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:30.762483 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e2112-dbee-457e-92c6-f81c3a8def9e None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:30.763159 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e2112-dbee-457e-92c6-f81c3a8def9e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:30.764257 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e2112-dbee-457e-92c6-f81c3a8def9e None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:30.766461 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 884/1802] 10.222.0.83 () {54 vars in 921 bytes} [Mon Aug 1 15:34:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:34:30.833389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944d68a-678a-4438-a6f3-edf78a17eafe None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:30.834301 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944d68a-678a-4438-a6f3-edf78a17eafe None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:30.834753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944d68a-678a-4438-a6f3-edf78a17eafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:30.852051 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042837c-70c3-4768-bc9c-b53e0ae6926b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:30.864240 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77834609-586f-4a2e-b0c9-b9803c8d13a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:30.868107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77834609-586f-4a2e-b0c9-b9803c8d13a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:30.871179 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77834609-586f-4a2e-b0c9-b9803c8d13a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:30.872005 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77834609-586f-4a2e-b0c9-b9803c8d13a7 admin admin] PATH_INFO: `/v3/projects/7d5edd8b3e49445187b9052fe50ec13a` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:30.881430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77834609-586f-4a2e-b0c9-b9803c8d13a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d5edd8b3e49445187b9052fe50ec13a)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:30.883688 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-5944d68a-678a-4438-a6f3-edf78a17eafe None None] MFA Rules not processed for user `ef57a9394b114eb3ab499133b5d40781`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:30.887626 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5944d68a-678a-4438-a6f3-edf78a17eafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:30.892822 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77834609-586f-4a2e-b0c9-b9803c8d13a7 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:31.012663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-77834609-586f-4a2e-b0c9-b9803c8d13a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d5edd8b3e49445187b9052fe50ec13a', 'request_id': 'req-77834609-586f-4a2e-b0c9-b9803c8d13a7'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:31.022843 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-77834609-586f-4a2e-b0c9-b9803c8d13a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d5edd8b3e49445187b9052fe50ec13a', 'request_id': 'req-77834609-586f-4a2e-b0c9-b9803c8d13a7'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:31.051886 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:31.072935 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-77834609-586f-4a2e-b0c9-b9803c8d13a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d5edd8b3e49445187b9052fe50ec13a', 'request_id': 'req-77834609-586f-4a2e-b0c9-b9803c8d13a7'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:31.106169 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-77834609-586f-4a2e-b0c9-b9803c8d13a7 admin admin] The token cache is being invalidate because project 7d5edd8b3e49445187b9052fe50ec13a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:31.110422 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-77834609-586f-4a2e-b0c9-b9803c8d13a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:31.112198 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-77834609-586f-4a2e-b0c9-b9803c8d13a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:31.113926 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 885/1803] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Aug 1 15:34:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:31.116358 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 919/1804] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:30 2022] DELETE /identity/v3/projects/7d5edd8b3e49445187b9052fe50ec13a => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:34:31.128143 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944d68a-678a-4438-a6f3-edf78a17eafe None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:31.147003 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944d68a-678a-4438-a6f3-edf78a17eafe None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:31.166003 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77834609-586f-4a2e-b0c9-b9803c8d13a7 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:31.169297 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-77834609-586f-4a2e-b0c9-b9803c8d13a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:31.195331 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:31.278042 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47b2c3-c55c-4db0-80a8-42878d257b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:31.279176 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47b2c3-c55c-4db0-80a8-42878d257b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:31.279598 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47b2c3-c55c-4db0-80a8-42878d257b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:31.279986 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47b2c3-c55c-4db0-80a8-42878d257b0c admin admin] PATH_INFO: `/v3/users/d3a6938502cc4777ab4c7d060dfb1586` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:31.281506 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47b2c3-c55c-4db0-80a8-42878d257b0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3a6938502cc4777ab4c7d060dfb1586)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:31.284556 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47b2c3-c55c-4db0-80a8-42878d257b0c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:31.392000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1f47b2c3-c55c-4db0-80a8-42878d257b0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3a6938502cc4777ab4c7d060dfb1586'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:31.392239 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77834609-586f-4a2e-b0c9-b9803c8d13a7 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:31.393253 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1f47b2c3-c55c-4db0-80a8-42878d257b0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3a6938502cc4777ab4c7d060dfb1586'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:31.408525 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1f47b2c3-c55c-4db0-80a8-42878d257b0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3a6938502cc4777ab4c7d060dfb1586'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:31.410065 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18e0e8-153d-48bf-8c60-8f0a5b80519e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:31.410735 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18e0e8-153d-48bf-8c60-8f0a5b80519e service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:31.411024 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18e0e8-153d-48bf-8c60-8f0a5b80519e service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:31.411306 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18e0e8-153d-48bf-8c60-8f0a5b80519e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:31.431675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2b18e0e8-153d-48bf-8c60-8f0a5b80519e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:31.440384 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1f47b2c3-c55c-4db0-80a8-42878d257b0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3a6938502cc4777ab4c7d060dfb1586'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:31.441300 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1f47b2c3-c55c-4db0-80a8-42878d257b0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3a6938502cc4777ab4c7d060dfb1586'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:31.469632 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18e0e8-153d-48bf-8c60-8f0a5b80519e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:31.471880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18e0e8-153d-48bf-8c60-8f0a5b80519e service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:31.474791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 886/1805] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:34:31 2022] DELETE /identity/v3/users/d3a6938502cc4777ab4c7d060dfb1586 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:31.476034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47b2c3-c55c-4db0-80a8-42878d257b0c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:31.480487 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1f47b2c3-c55c-4db0-80a8-42878d257b0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:31.494731 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 920/1806] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:34:31.630560 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47b2c3-c55c-4db0-80a8-42878d257b0c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:31.638043 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f479a1-fa39-4e7e-9898-7391847dd43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:31.638888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f479a1-fa39-4e7e-9898-7391847dd43d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:31.639313 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f479a1-fa39-4e7e-9898-7391847dd43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:31.640491 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f479a1-fa39-4e7e-9898-7391847dd43d admin admin] PATH_INFO: `/v3/users/79cefb8ac83f470e86d2f28845854d18` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:31.642150 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f479a1-fa39-4e7e-9898-7391847dd43d admin admin] RBAC: Authorizing `identity:delete_user(user_id=79cefb8ac83f470e86d2f28845854d18)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:31.645208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f479a1-fa39-4e7e-9898-7391847dd43d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:31.706550 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:31.790299 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b3f479a1-fa39-4e7e-9898-7391847dd43d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79cefb8ac83f470e86d2f28845854d18'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:31.792816 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b3f479a1-fa39-4e7e-9898-7391847dd43d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79cefb8ac83f470e86d2f28845854d18'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:31.805826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b3f479a1-fa39-4e7e-9898-7391847dd43d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79cefb8ac83f470e86d2f28845854d18'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:31.829232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b3f479a1-fa39-4e7e-9898-7391847dd43d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79cefb8ac83f470e86d2f28845854d18'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:31.829949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b3f479a1-fa39-4e7e-9898-7391847dd43d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79cefb8ac83f470e86d2f28845854d18'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:31.867022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 887/1807] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:34:31 2022] DELETE /identity/v3/users/79cefb8ac83f470e86d2f28845854d18 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:32.534718 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18e0e8-153d-48bf-8c60-8f0a5b80519e service placement] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:32.536209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2b18e0e8-153d-48bf-8c60-8f0a5b80519e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:32.558339 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f479a1-fa39-4e7e-9898-7391847dd43d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:32.560278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b3f479a1-fa39-4e7e-9898-7391847dd43d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:32.564138 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18e0e8-153d-48bf-8c60-8f0a5b80519e service placement] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:32.669155 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5c0d8-975b-49ac-8843-07113fea0ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:32.670599 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5c0d8-975b-49ac-8843-07113fea0ce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:32.670987 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5c0d8-975b-49ac-8843-07113fea0ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:32.671368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5c0d8-975b-49ac-8843-07113fea0ce2 admin admin] PATH_INFO: `/v3/projects/a7936102185d4a7f994d337368487582` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:32.673445 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d5c0d8-975b-49ac-8843-07113fea0ce2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7936102185d4a7f994d337368487582)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:32.676894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d5c0d8-975b-49ac-8843-07113fea0ce2 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:32.696399 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f479a1-fa39-4e7e-9898-7391847dd43d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:32.703317 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c568b3-a845-4ee4-9559-9eb1c773f312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:32.704095 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c568b3-a845-4ee4-9559-9eb1c773f312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:32.704609 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c568b3-a845-4ee4-9559-9eb1c773f312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:32.705208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c568b3-a845-4ee4-9559-9eb1c773f312 admin admin] PATH_INFO: `/v3/projects/e9d1933937804b3595b8f92a990c1f9b` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:32.707448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c568b3-a845-4ee4-9559-9eb1c773f312 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9d1933937804b3595b8f92a990c1f9b)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:32.710012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c568b3-a845-4ee4-9559-9eb1c773f312 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:32.779198 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b5d5c0d8-975b-49ac-8843-07113fea0ce2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7936102185d4a7f994d337368487582', 'request_id': 'req-b5d5c0d8-975b-49ac-8843-07113fea0ce2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:32.793471 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b5d5c0d8-975b-49ac-8843-07113fea0ce2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7936102185d4a7f994d337368487582', 'request_id': 'req-b5d5c0d8-975b-49ac-8843-07113fea0ce2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:32.806672 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b5d5c0d8-975b-49ac-8843-07113fea0ce2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7936102185d4a7f994d337368487582', 'request_id': 'req-b5d5c0d8-975b-49ac-8843-07113fea0ce2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:32.813779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-50c568b3-a845-4ee4-9559-9eb1c773f312 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9d1933937804b3595b8f92a990c1f9b', 'request_id': 'req-50c568b3-a845-4ee4-9559-9eb1c773f312'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:32.834558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-50c568b3-a845-4ee4-9559-9eb1c773f312 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9d1933937804b3595b8f92a990c1f9b', 'request_id': 'req-50c568b3-a845-4ee4-9559-9eb1c773f312'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:32.854070 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-50c568b3-a845-4ee4-9559-9eb1c773f312 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9d1933937804b3595b8f92a990c1f9b', 'request_id': 'req-50c568b3-a845-4ee4-9559-9eb1c773f312'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:32.855410 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b5d5c0d8-975b-49ac-8843-07113fea0ce2 admin admin] The token cache is being invalidate because project a7936102185d4a7f994d337368487582 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:32.858429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b5d5c0d8-975b-49ac-8843-07113fea0ce2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:32.861534 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b5d5c0d8-975b-49ac-8843-07113fea0ce2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:32.867788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 921/1808] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:32 2022] DELETE /identity/v3/projects/a7936102185d4a7f994d337368487582 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:32.869644 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1bd4f-7363-4d36-8fdd-8727047844b0 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:32.869974 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1bd4f-7363-4d36-8fdd-8727047844b0 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:32.874652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1bd4f-7363-4d36-8fdd-8727047844b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:32.888256 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-50c568b3-a845-4ee4-9559-9eb1c773f312 admin admin] The token cache is being invalidate because project e9d1933937804b3595b8f92a990c1f9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:32.888804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-50c568b3-a845-4ee4-9559-9eb1c773f312 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:32.892581 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-50c568b3-a845-4ee4-9559-9eb1c773f312 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:32.921862 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 888/1809] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:32 2022] DELETE /identity/v3/projects/e9d1933937804b3595b8f92a990c1f9b => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:34:32.921862 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c568b3-a845-4ee4-9559-9eb1c773f312 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:32.921862 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-50c568b3-a845-4ee4-9559-9eb1c773f312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:32.944685 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-08b1bd4f-7363-4d36-8fdd-8727047844b0 None None] MFA Rules not processed for user `8daf22a6b42844b3a8db7affe1016a84`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:32.946067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-08b1bd4f-7363-4d36-8fdd-8727047844b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:32.961567 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:32.979850 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:33.064974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c568b3-a845-4ee4-9559-9eb1c773f312 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:33.073369 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95b216-ecb3-46ee-b593-f71a4386cbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:33.074117 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95b216-ecb3-46ee-b593-f71a4386cbb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:33.074404 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95b216-ecb3-46ee-b593-f71a4386cbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:33.074681 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95b216-ecb3-46ee-b593-f71a4386cbb4 admin admin] PATH_INFO: `/v3/users/55996ae2c02840d698b422f0dc9ad22d` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:33.077712 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95b216-ecb3-46ee-b593-f71a4386cbb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55996ae2c02840d698b422f0dc9ad22d)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:33.080059 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95b216-ecb3-46ee-b593-f71a4386cbb4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:33.116806 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 922/1810] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:33.125710 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d19777-364e-427c-916c-8ff9d15f24ee None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:33.125978 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d19777-364e-427c-916c-8ff9d15f24ee None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:33.126263 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d19777-364e-427c-916c-8ff9d15f24ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:33.207774 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-54d19777-364e-427c-916c-8ff9d15f24ee None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:33.226629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-54d19777-364e-427c-916c-8ff9d15f24ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:33.249089 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2b95b216-ecb3-46ee-b593-f71a4386cbb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55996ae2c02840d698b422f0dc9ad22d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:33.250239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2b95b216-ecb3-46ee-b593-f71a4386cbb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55996ae2c02840d698b422f0dc9ad22d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:33.260972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 923/1811] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:33.263752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2b95b216-ecb3-46ee-b593-f71a4386cbb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55996ae2c02840d698b422f0dc9ad22d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:33.265178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d19777-364e-427c-916c-8ff9d15f24ee None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:33.272336 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-54d19777-364e-427c-916c-8ff9d15f24ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:33.304809 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2b95b216-ecb3-46ee-b593-f71a4386cbb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55996ae2c02840d698b422f0dc9ad22d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:33.306645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2b95b216-ecb3-46ee-b593-f71a4386cbb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55996ae2c02840d698b422f0dc9ad22d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:33.342428 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 889/1812] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:34:32 2022] DELETE /identity/v3/users/55996ae2c02840d698b422f0dc9ad22d => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:33.344658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da3515-5f22-4766-9d7a-37a5c7d78901 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:33.344967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da3515-5f22-4766-9d7a-37a5c7d78901 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:33.345250 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da3515-5f22-4766-9d7a-37a5c7d78901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:33.370244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-b3da3515-5f22-4766-9d7a-37a5c7d78901 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:33.371634 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b3da3515-5f22-4766-9d7a-37a5c7d78901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:33.427668 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d19777-364e-427c-916c-8ff9d15f24ee None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:33.490680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 890/1813] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:33.494460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da3515-5f22-4766-9d7a-37a5c7d78901 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:33.510360 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da3515-5f22-4766-9d7a-37a5c7d78901 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:33.521235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad10c1e-4fde-4323-ba17-da8f8446353a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:33.524749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad10c1e-4fde-4323-ba17-da8f8446353a admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:33.524749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad10c1e-4fde-4323-ba17-da8f8446353a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:33.524749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad10c1e-4fde-4323-ba17-da8f8446353a admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:33.524749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad10c1e-4fde-4323-ba17-da8f8446353a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:34:33.524749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad10c1e-4fde-4323-ba17-da8f8446353a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:33.527335 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad10c1e-4fde-4323-ba17-da8f8446353a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:33.532811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186b831-9eb3-4ee1-b194-044402d0e7dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:33.533458 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186b831-9eb3-4ee1-b194-044402d0e7dc service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:33.533744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186b831-9eb3-4ee1-b194-044402d0e7dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:33.534112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186b831-9eb3-4ee1-b194-044402d0e7dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:33.536140 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b186b831-9eb3-4ee1-b194-044402d0e7dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:33.549727 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 891/1814] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:34:33.554043 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad10c1e-4fde-4323-ba17-da8f8446353a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:33.573371 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad10c1e-4fde-4323-ba17-da8f8446353a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:33.581693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e9968-7b01-465d-bbe1-dc222f703043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:33.582800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e9968-7b01-465d-bbe1-dc222f703043 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:33.583185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e9968-7b01-465d-bbe1-dc222f703043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:33.583673 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e9968-7b01-465d-bbe1-dc222f703043 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:33.586272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e9968-7b01-465d-bbe1-dc222f703043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:33.587944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e9968-7b01-465d-bbe1-dc222f703043 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:33.644347 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 892/1815] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:34:33 2022] POST /identity/v3/projects => generated 401 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:33.649802 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e9968-7b01-465d-bbe1-dc222f703043 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:33.665570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186b831-9eb3-4ee1-b194-044402d0e7dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:33.665787 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:33.667082 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186b831-9eb3-4ee1-b194-044402d0e7dc service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:33.699441 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 924/1816] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:33.787896 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e9968-7b01-465d-bbe1-dc222f703043 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:33.804247 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fb664-1415-4809-9242-5236069b67dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:33.804988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fb664-1415-4809-9242-5236069b67dc admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:33.805387 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fb664-1415-4809-9242-5236069b67dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:33.805776 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fb664-1415-4809-9242-5236069b67dc admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:33.806609 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fb664-1415-4809-9242-5236069b67dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:33.808748 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fb664-1415-4809-9242-5236069b67dc admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:33.854104 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 893/1817] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:34:33 2022] POST /identity/v3/users => generated 301 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:33.857871 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186b831-9eb3-4ee1-b194-044402d0e7dc service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:33.876615 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186b831-9eb3-4ee1-b194-044402d0e7dc service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:33.882843 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48362451-3f27-4bd3-9368-01c7c5fde82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:33.883997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48362451-3f27-4bd3-9368-01c7c5fde82f admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:33.884565 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48362451-3f27-4bd3-9368-01c7c5fde82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:33.885171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48362451-3f27-4bd3-9368-01c7c5fde82f admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:33.886085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48362451-3f27-4bd3-9368-01c7c5fde82f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:33.887898 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48362451-3f27-4bd3-9368-01c7c5fde82f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:33.904569 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 925/1818] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:33.909667 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fb664-1415-4809-9242-5236069b67dc admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:33.931101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fb664-1415-4809-9242-5236069b67dc admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:33.947763 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f22d0-7530-40ea-9f2b-c58c9df5dae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:33.948401 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f22d0-7530-40ea-9f2b-c58c9df5dae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:33.948885 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f22d0-7530-40ea-9f2b-c58c9df5dae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:33.949347 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f22d0-7530-40ea-9f2b-c58c9df5dae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:33.950578 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f22d0-7530-40ea-9f2b-c58c9df5dae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:33.953680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f22d0-7530-40ea-9f2b-c58c9df5dae3 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:33.990314 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 894/1819] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:34.002693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48362451-3f27-4bd3-9368-01c7c5fde82f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:34.019383 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:34.025361 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48362451-3f27-4bd3-9368-01c7c5fde82f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:34.041758 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab4256-7792-43f0-935f-3d4021d85d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:34.042060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab4256-7792-43f0-935f-3d4021d85d14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:34.042634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab4256-7792-43f0-935f-3d4021d85d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:34.043101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab4256-7792-43f0-935f-3d4021d85d14 admin admin] PATH_INFO: `/v3/projects/f972d0bf875c463da84a13f3ccb3eea2/users/31da4d2a391f4e879d6a2960f2bd7048/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:34.128950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab4256-7792-43f0-935f-3d4021d85d14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f972d0bf875c463da84a13f3ccb3eea2, user_id=31da4d2a391f4e879d6a2960f2bd7048, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:34.146164 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab4256-7792-43f0-935f-3d4021d85d14 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:34.206527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 926/1820] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:33 2022] PUT /identity/v3/projects/f972d0bf875c463da84a13f3ccb3eea2/users/31da4d2a391f4e879d6a2960f2bd7048/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:34.223127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3366f-a388-44d7-8171-dde89fe95c3e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:34.223731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3366f-a388-44d7-8171-dde89fe95c3e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:34.224158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3366f-a388-44d7-8171-dde89fe95c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:34.284907 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4bd3366f-a388-44d7-8171-dde89fe95c3e None None] MFA Rules not processed for user `31da4d2a391f4e879d6a2960f2bd7048`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:34.288270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4bd3366f-a388-44d7-8171-dde89fe95c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:34.420525 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 895/1821] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:34:34.423264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab4256-7792-43f0-935f-3d4021d85d14 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:34.523936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab4256-7792-43f0-935f-3d4021d85d14 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:34.536387 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee198ae-830a-4472-82fe-22b3163702f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:34.537446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee198ae-830a-4472-82fe-22b3163702f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:34.537446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee198ae-830a-4472-82fe-22b3163702f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:34.537446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee198ae-830a-4472-82fe-22b3163702f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:34.538711 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee198ae-830a-4472-82fe-22b3163702f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:34.543843 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee198ae-830a-4472-82fe-22b3163702f7 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:34.596922 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd3366f-a388-44d7-8171-dde89fe95c3e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:34.597554 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 927/1822] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:34:34 2022] POST /identity/v3/projects => generated 399 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:34.619471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd3366f-a388-44d7-8171-dde89fe95c3e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:34.724258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557202d2-3bd1-4288-9ad0-0835f6c1c92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:34.724990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557202d2-3bd1-4288-9ad0-0835f6c1c92f admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:34.726672 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557202d2-3bd1-4288-9ad0-0835f6c1c92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:34.727681 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557202d2-3bd1-4288-9ad0-0835f6c1c92f admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:34.729073 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557202d2-3bd1-4288-9ad0-0835f6c1c92f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:34.733916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557202d2-3bd1-4288-9ad0-0835f6c1c92f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:34.774528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 896/1823] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:34:34 2022] POST /identity/v3/users => generated 306 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:34.780481 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee198ae-830a-4472-82fe-22b3163702f7 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:34.797454 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee198ae-830a-4472-82fe-22b3163702f7 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:34.804857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdd75e-f234-4cdc-aa69-09d2703d3f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:34.805737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdd75e-f234-4cdc-aa69-09d2703d3f97 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:34.806203 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdd75e-f234-4cdc-aa69-09d2703d3f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:34.806558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdd75e-f234-4cdc-aa69-09d2703d3f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:34.807437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cdd75e-f234-4cdc-aa69-09d2703d3f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:34.809164 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cdd75e-f234-4cdc-aa69-09d2703d3f97 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:34.832189 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 928/1824] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:34.836628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557202d2-3bd1-4288-9ad0-0835f6c1c92f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:34.851829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:34.855722 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557202d2-3bd1-4288-9ad0-0835f6c1c92f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:34.890616 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54a1c9-6bfa-48e7-baa9-72f269b6d207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:34.891456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54a1c9-6bfa-48e7-baa9-72f269b6d207 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:34.891801 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54a1c9-6bfa-48e7-baa9-72f269b6d207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:34.892100 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54a1c9-6bfa-48e7-baa9-72f269b6d207 admin admin] PATH_INFO: `/v3/projects/ff6020c9a74549ff9240edbc6964e352/users/e0c783cc45b04ea3aec1ba97980a6f67/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:34.920373 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdd75e-f234-4cdc-aa69-09d2703d3f97 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:34.922467 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-31cdd75e-f234-4cdc-aa69-09d2703d3f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:34.957349 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54a1c9-6bfa-48e7-baa9-72f269b6d207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff6020c9a74549ff9240edbc6964e352, user_id=e0c783cc45b04ea3aec1ba97980a6f67, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:34.958763 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdd75e-f234-4cdc-aa69-09d2703d3f97 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:34.962515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54a1c9-6bfa-48e7-baa9-72f269b6d207 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:34.985584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d424a1-368e-4f7a-85d6-f275c82a468b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:34.986346 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d424a1-368e-4f7a-85d6-f275c82a468b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:34.986629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d424a1-368e-4f7a-85d6-f275c82a468b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:34.989339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d424a1-368e-4f7a-85d6-f275c82a468b admin admin] PATH_INFO: `/v3/projects/28e4231ebaf344a9928809d01227ead1` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:34.990388 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 897/1825] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:34 2022] PUT /identity/v3/projects/ff6020c9a74549ff9240edbc6964e352/users/e0c783cc45b04ea3aec1ba97980a6f67/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:34.994310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54a1c9-6bfa-48e7-baa9-72f269b6d207 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:34.995169 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d424a1-368e-4f7a-85d6-f275c82a468b admin admin] RBAC: Authorizing `identity:delete_project(project_id=28e4231ebaf344a9928809d01227ead1)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:35.003912 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d424a1-368e-4f7a-85d6-f275c82a468b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:35.097632 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54a1c9-6bfa-48e7-baa9-72f269b6d207 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:35.136895 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-90d424a1-368e-4f7a-85d6-f275c82a468b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28e4231ebaf344a9928809d01227ead1', 'request_id': 'req-90d424a1-368e-4f7a-85d6-f275c82a468b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:35.159003 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-90d424a1-368e-4f7a-85d6-f275c82a468b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28e4231ebaf344a9928809d01227ead1', 'request_id': 'req-90d424a1-368e-4f7a-85d6-f275c82a468b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:35.169555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-90d424a1-368e-4f7a-85d6-f275c82a468b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28e4231ebaf344a9928809d01227ead1', 'request_id': 'req-90d424a1-368e-4f7a-85d6-f275c82a468b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:35.174288 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ded1c5-e670-48d7-bbc1-1318f2c7a0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:35.175109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ded1c5-e670-48d7-bbc1-1318f2c7a0a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:35.175380 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ded1c5-e670-48d7-bbc1-1318f2c7a0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:35.175655 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ded1c5-e670-48d7-bbc1-1318f2c7a0a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:35.176434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ded1c5-e670-48d7-bbc1-1318f2c7a0a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:35.178456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ded1c5-e670-48d7-bbc1-1318f2c7a0a8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:35.192806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 898/1826] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:35.196955 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ded1c5-e670-48d7-bbc1-1318f2c7a0a8 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:35.202874 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-90d424a1-368e-4f7a-85d6-f275c82a468b admin admin] The token cache is being invalidate because project 28e4231ebaf344a9928809d01227ead1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:35.203244 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-90d424a1-368e-4f7a-85d6-f275c82a468b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:35.204010 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-90d424a1-368e-4f7a-85d6-f275c82a468b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:35.209315 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 929/1827] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:34 2022] DELETE /identity/v3/projects/28e4231ebaf344a9928809d01227ead1 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:34:35.214699 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9a58d-8bd5-4845-ac46-e46ca33f23e8 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:35.215000 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9a58d-8bd5-4845-ac46-e46ca33f23e8 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:35.215329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9a58d-8bd5-4845-ac46-e46ca33f23e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:35.218388 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ded1c5-e670-48d7-bbc1-1318f2c7a0a8 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:35.226489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c381127-7fcf-4237-b33b-2016b0ad5642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:35.228443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c381127-7fcf-4237-b33b-2016b0ad5642 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:35.228833 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c381127-7fcf-4237-b33b-2016b0ad5642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:35.229158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c381127-7fcf-4237-b33b-2016b0ad5642 admin admin] PATH_INFO: `/v3/projects/ff6020c9a74549ff9240edbc6964e352/users/e0c783cc45b04ea3aec1ba97980a6f67/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:35.231732 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c381127-7fcf-4237-b33b-2016b0ad5642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff6020c9a74549ff9240edbc6964e352, user_id=e0c783cc45b04ea3aec1ba97980a6f67, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:35.235879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c381127-7fcf-4237-b33b-2016b0ad5642 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:35.252230 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 899/1828] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:35 2022] PUT /identity/v3/projects/ff6020c9a74549ff9240edbc6964e352/users/e0c783cc45b04ea3aec1ba97980a6f67/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:35.269291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7d5ae-15b5-4ff4-95f2-160cd38536df None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:35.269710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7d5ae-15b5-4ff4-95f2-160cd38536df None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:35.270022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7d5ae-15b5-4ff4-95f2-160cd38536df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:35.286965 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2fe9a58d-8bd5-4845-ac46-e46ca33f23e8 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:35.288292 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2fe9a58d-8bd5-4845-ac46-e46ca33f23e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:35.349290 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-40c7d5ae-15b5-4ff4-95f2-160cd38536df None None] MFA Rules not processed for user `e0c783cc45b04ea3aec1ba97980a6f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:35.350734 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-40c7d5ae-15b5-4ff4-95f2-160cd38536df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:35.384052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:35.492472 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 930/1829] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:35.492472 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9a58d-8bd5-4845-ac46-e46ca33f23e8 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:35.493807 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2fe9a58d-8bd5-4845-ac46-e46ca33f23e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:35.581074 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:35.604735 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 900/1830] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:35 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:35.606438 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f95b72-ac05-42f3-aaef-1751b9f61c62 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:35.606800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f95b72-ac05-42f3-aaef-1751b9f61c62 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:35.607675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f95b72-ac05-42f3-aaef-1751b9f61c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:35.658236 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9a58d-8bd5-4845-ac46-e46ca33f23e8 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:35.667840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45abae-d8d1-48c5-a617-ac90df7e35ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:35.669268 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45abae-d8d1-48c5-a617-ac90df7e35ab service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:35.670596 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45abae-d8d1-48c5-a617-ac90df7e35ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:35.671816 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45abae-d8d1-48c5-a617-ac90df7e35ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:35.681655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3f45abae-d8d1-48c5-a617-ac90df7e35ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:35.696577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e7f95b72-ac05-42f3-aaef-1751b9f61c62 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:35.698498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e7f95b72-ac05-42f3-aaef-1751b9f61c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:35.714328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 901/1831] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:35.715251 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f95b72-ac05-42f3-aaef-1751b9f61c62 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:35.719144 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e7f95b72-ac05-42f3-aaef-1751b9f61c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:35.747578 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f95b72-ac05-42f3-aaef-1751b9f61c62 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:35.759529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7636b7-58fc-4879-839d-20fc32616c03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:35.760623 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7636b7-58fc-4879-839d-20fc32616c03 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:35.760623 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7636b7-58fc-4879-839d-20fc32616c03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:35.760941 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7636b7-58fc-4879-839d-20fc32616c03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:35.787026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7636b7-58fc-4879-839d-20fc32616c03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:35.793148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7636b7-58fc-4879-839d-20fc32616c03 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:35.823979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45abae-d8d1-48c5-a617-ac90df7e35ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:35.825759 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45abae-d8d1-48c5-a617-ac90df7e35ab service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:35.827312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 902/1832] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:35 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:35.828549 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7636b7-58fc-4879-839d-20fc32616c03 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:35.844881 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 931/1833] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:35 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:35.887826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7636b7-58fc-4879-839d-20fc32616c03 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:35.915885 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571847c-6893-4cf0-a2e4-9719fc7e7b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:35.916543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571847c-6893-4cf0-a2e4-9719fc7e7b53 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:35.916832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571847c-6893-4cf0-a2e4-9719fc7e7b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:35.917104 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571847c-6893-4cf0-a2e4-9719fc7e7b53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:35.917821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571847c-6893-4cf0-a2e4-9719fc7e7b53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:34:35.919331 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571847c-6893-4cf0-a2e4-9719fc7e7b53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:35.929698 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571847c-6893-4cf0-a2e4-9719fc7e7b53 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:35.985168 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 903/1834] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:34:35.989383 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45abae-d8d1-48c5-a617-ac90df7e35ab service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:35.993078 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571847c-6893-4cf0-a2e4-9719fc7e7b53 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:36.005771 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571847c-6893-4cf0-a2e4-9719fc7e7b53 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:36.008593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45abae-d8d1-48c5-a617-ac90df7e35ab service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:36.012008 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f149e75-7e9a-40a1-9e45-645ca478d479 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:36.013186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f149e75-7e9a-40a1-9e45-645ca478d479 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:36.013661 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f149e75-7e9a-40a1-9e45-645ca478d479 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:36.014435 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f149e75-7e9a-40a1-9e45-645ca478d479 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:36.016482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8f149e75-7e9a-40a1-9e45-645ca478d479 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:36.025817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0254a-3dbb-4206-8222-2fddfcd931f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:36.026601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0254a-3dbb-4206-8222-2fddfcd931f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:36.028242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0254a-3dbb-4206-8222-2fddfcd931f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:36.028645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0254a-3dbb-4206-8222-2fddfcd931f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:36.031721 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0254a-3dbb-4206-8222-2fddfcd931f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:36.037713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0254a-3dbb-4206-8222-2fddfcd931f6 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:36.124426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 932/1835] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:34:35 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:36.125793 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0254a-3dbb-4206-8222-2fddfcd931f6 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:36.252552 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f149e75-7e9a-40a1-9e45-645ca478d479 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:36.264724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f149e75-7e9a-40a1-9e45-645ca478d479 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:36.344144 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0254a-3dbb-4206-8222-2fddfcd931f6 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:36.346132 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 904/1836] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:35 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:36.362874 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89d5b2-b254-4e93-ac45-69043664ebfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:36.363535 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89d5b2-b254-4e93-ac45-69043664ebfc admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:36.363820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89d5b2-b254-4e93-ac45-69043664ebfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:36.364100 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89d5b2-b254-4e93-ac45-69043664ebfc admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:36.364797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c89d5b2-b254-4e93-ac45-69043664ebfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:36.367482 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c89d5b2-b254-4e93-ac45-69043664ebfc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:36.399597 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:36.419015 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 933/1837] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:34:36 2022] POST /identity/v3/users => generated 294 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:34:36.424509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f149e75-7e9a-40a1-9e45-645ca478d479 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:36.445464 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f149e75-7e9a-40a1-9e45-645ca478d479 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:36.464661 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4a21e-49e1-408f-b6cc-1e4751f10776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:36.465509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4a21e-49e1-408f-b6cc-1e4751f10776 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:36.465981 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4a21e-49e1-408f-b6cc-1e4751f10776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:36.466404 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4a21e-49e1-408f-b6cc-1e4751f10776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:36.467830 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4a21e-49e1-408f-b6cc-1e4751f10776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:36.473520 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4a21e-49e1-408f-b6cc-1e4751f10776 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:36.517761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 905/1838] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:36.521740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89d5b2-b254-4e93-ac45-69043664ebfc admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:36.556953 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89d5b2-b254-4e93-ac45-69043664ebfc admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:36.568125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de1603-d9bc-4439-8d55-b1ce1c78878c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:36.570355 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de1603-d9bc-4439-8d55-b1ce1c78878c admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:36.571582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de1603-d9bc-4439-8d55-b1ce1c78878c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:36.573394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de1603-d9bc-4439-8d55-b1ce1c78878c admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:36.574305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de1603-d9bc-4439-8d55-b1ce1c78878c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:36.578005 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de1603-d9bc-4439-8d55-b1ce1c78878c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:36.593932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 934/1839] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:36.598766 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4a21e-49e1-408f-b6cc-1e4751f10776 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:36.613228 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4a21e-49e1-408f-b6cc-1e4751f10776 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:36.624962 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed623883-6626-47a5-a122-632d66ea1be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:36.626137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed623883-6626-47a5-a122-632d66ea1be2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:36.627273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed623883-6626-47a5-a122-632d66ea1be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:36.627273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed623883-6626-47a5-a122-632d66ea1be2 admin admin] PATH_INFO: `/v3/projects/bbd1adcd43054b50bb8a5d41a27d44bc/users/5e4dd4d557084f65b5f9193e1433045d/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:36.672736 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed623883-6626-47a5-a122-632d66ea1be2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbd1adcd43054b50bb8a5d41a27d44bc, user_id=5e4dd4d557084f65b5f9193e1433045d, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:36.678608 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed623883-6626-47a5-a122-632d66ea1be2 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:36.714116 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 906/1840] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:36 2022] PUT /identity/v3/projects/bbd1adcd43054b50bb8a5d41a27d44bc/users/5e4dd4d557084f65b5f9193e1433045d/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:36.722590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd522f-f099-4bb3-93df-0d3c50cde91b None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:36.723269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd522f-f099-4bb3-93df-0d3c50cde91b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:36.723269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd522f-f099-4bb3-93df-0d3c50cde91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:36.765270 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-ebfd522f-f099-4bb3-93df-0d3c50cde91b None None] MFA Rules not processed for user `5e4dd4d557084f65b5f9193e1433045d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:36.766783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ebfd522f-f099-4bb3-93df-0d3c50cde91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:36.872503 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 935/1841] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:36.879344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed623883-6626-47a5-a122-632d66ea1be2 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:37.026143 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed623883-6626-47a5-a122-632d66ea1be2 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:37.039628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708876b8-a9ec-46cc-a724-9010e9b4557b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:37.041391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708876b8-a9ec-46cc-a724-9010e9b4557b admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:37.048411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708876b8-a9ec-46cc-a724-9010e9b4557b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:37.048832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708876b8-a9ec-46cc-a724-9010e9b4557b admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:37.050272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708876b8-a9ec-46cc-a724-9010e9b4557b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:37.052394 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708876b8-a9ec-46cc-a724-9010e9b4557b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:37.123584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 907/1842] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:34:36 2022] POST /identity/v3/projects => generated 387 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:37.137981 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd522f-f099-4bb3-93df-0d3c50cde91b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:37.163211 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd522f-f099-4bb3-93df-0d3c50cde91b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:37.271782 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ec711-9f07-4cd6-9a54-a766522e9e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:37.272162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ec711-9f07-4cd6-9a54-a766522e9e0e admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:37.272443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ec711-9f07-4cd6-9a54-a766522e9e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:37.272713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ec711-9f07-4cd6-9a54-a766522e9e0e admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:37.273457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ec711-9f07-4cd6-9a54-a766522e9e0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:37.275429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ec711-9f07-4cd6-9a54-a766522e9e0e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:37.308639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 936/1843] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:34:37 2022] POST /identity/v3/users => generated 300 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:37.313391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708876b8-a9ec-46cc-a724-9010e9b4557b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:37.359672 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708876b8-a9ec-46cc-a724-9010e9b4557b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:37.367864 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0ac1e-6ecd-4003-a4a3-4e23c1bca62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:37.368665 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0ac1e-6ecd-4003-a4a3-4e23c1bca62c admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:37.368774 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0ac1e-6ecd-4003-a4a3-4e23c1bca62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:37.369035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0ac1e-6ecd-4003-a4a3-4e23c1bca62c admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:37.369771 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0ac1e-6ecd-4003-a4a3-4e23c1bca62c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:37.371482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0ac1e-6ecd-4003-a4a3-4e23c1bca62c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:37.383708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:37.403395 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 908/1844] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:37.403730 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ec711-9f07-4cd6-9a54-a766522e9e0e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:37.415236 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ec711-9f07-4cd6-9a54-a766522e9e0e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:37.444709 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4497e99-8ca9-4ff6-8d3c-5948600eef0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:37.445570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4497e99-8ca9-4ff6-8d3c-5948600eef0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:37.445973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4497e99-8ca9-4ff6-8d3c-5948600eef0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:37.446324 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4497e99-8ca9-4ff6-8d3c-5948600eef0b admin admin] PATH_INFO: `/v3/projects/6af33169e54d479c88294a7b8e1bb166/users/b3510304f4d346b1bc7a1f0f65a0a91a/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:37.467936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4497e99-8ca9-4ff6-8d3c-5948600eef0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6af33169e54d479c88294a7b8e1bb166, user_id=b3510304f4d346b1bc7a1f0f65a0a91a, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:37.470734 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4497e99-8ca9-4ff6-8d3c-5948600eef0b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:37.494654 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 937/1845] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:37 2022] PUT /identity/v3/projects/6af33169e54d479c88294a7b8e1bb166/users/b3510304f4d346b1bc7a1f0f65a0a91a/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:37.496309 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0ac1e-6ecd-4003-a4a3-4e23c1bca62c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:37.511333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0ac1e-6ecd-4003-a4a3-4e23c1bca62c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:37.620946 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150a866-264a-46ea-9008-6ee7ac552c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:37.626901 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150a866-264a-46ea-9008-6ee7ac552c44 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:37.627277 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150a866-264a-46ea-9008-6ee7ac552c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:37.627631 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150a866-264a-46ea-9008-6ee7ac552c44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:37.630993 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150a866-264a-46ea-9008-6ee7ac552c44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:37.634356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150a866-264a-46ea-9008-6ee7ac552c44 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:37.668074 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 909/1846] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:37.672901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4497e99-8ca9-4ff6-8d3c-5948600eef0b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:37.698129 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4497e99-8ca9-4ff6-8d3c-5948600eef0b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:37.715589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb874c-b434-41b4-9bbf-4155a66b7f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:37.732799 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb874c-b434-41b4-9bbf-4155a66b7f25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:37.733770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb874c-b434-41b4-9bbf-4155a66b7f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:37.733770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb874c-b434-41b4-9bbf-4155a66b7f25 admin admin] PATH_INFO: `/v3/projects/6af33169e54d479c88294a7b8e1bb166/users/b3510304f4d346b1bc7a1f0f65a0a91a/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:37.736582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb874c-b434-41b4-9bbf-4155a66b7f25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6af33169e54d479c88294a7b8e1bb166, user_id=b3510304f4d346b1bc7a1f0f65a0a91a, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:37.740509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb874c-b434-41b4-9bbf-4155a66b7f25 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:37.757623 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 938/1847] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:37 2022] PUT /identity/v3/projects/6af33169e54d479c88294a7b8e1bb166/users/b3510304f4d346b1bc7a1f0f65a0a91a/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:34:37.768367 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7ad92-e8bd-48ac-8cab-15b3487eca91 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:37.769008 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7ad92-e8bd-48ac-8cab-15b3487eca91 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:37.769439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7ad92-e8bd-48ac-8cab-15b3487eca91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:37.841877 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e4b7ad92-e8bd-48ac-8cab-15b3487eca91 None None] MFA Rules not processed for user `b3510304f4d346b1bc7a1f0f65a0a91a`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:37.843202 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e4b7ad92-e8bd-48ac-8cab-15b3487eca91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:38.009141 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 910/1848] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:38.026502 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb874c-b434-41b4-9bbf-4155a66b7f25 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:38.050666 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c3644-362a-4772-ac7e-e384d8996d0d None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:38.050960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c3644-362a-4772-ac7e-e384d8996d0d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:38.051252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c3644-362a-4772-ac7e-e384d8996d0d None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:38.053000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 911/1849] 10.222.0.83 () {54 vars in 921 bytes} [Mon Aug 1 15:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:34:38.075005 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165daeb-66ce-4f73-aa4c-ee32e820fc1d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:38.075278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165daeb-66ce-4f73-aa4c-ee32e820fc1d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:38.075591 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165daeb-66ce-4f73-aa4c-ee32e820fc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:38.156603 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a165daeb-66ce-4f73-aa4c-ee32e820fc1d None None] MFA Rules not processed for user `25dc41ffc41c47fdbeec06110d04610c`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:38.158083 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a165daeb-66ce-4f73-aa4c-ee32e820fc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:38.183660 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:38.267969 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb874c-b434-41b4-9bbf-4155a66b7f25 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:38.280827 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36acf764-809b-44fc-a1b8-f20f77268b37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:38.281493 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acf764-809b-44fc-a1b8-f20f77268b37 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:38.281870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acf764-809b-44fc-a1b8-f20f77268b37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:38.282172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acf764-809b-44fc-a1b8-f20f77268b37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:38.318018 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36acf764-809b-44fc-a1b8-f20f77268b37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:38.319478 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36acf764-809b-44fc-a1b8-f20f77268b37 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:38.327775 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 912/1850] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Aug 1 15:34:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:34:38.363691 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 939/1851] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:38.405702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165daeb-66ce-4f73-aa4c-ee32e820fc1d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:38.444408 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:38.534979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36acf764-809b-44fc-a1b8-f20f77268b37 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:38.536469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-36acf764-809b-44fc-a1b8-f20f77268b37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:38.573074 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36acf764-809b-44fc-a1b8-f20f77268b37 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:38.598588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed120f-e822-445b-bda1-39584289b982 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:38.599995 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed120f-e822-445b-bda1-39584289b982 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:38.600616 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed120f-e822-445b-bda1-39584289b982 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:38.600897 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed120f-e822-445b-bda1-39584289b982 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:38.610767 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165daeb-66ce-4f73-aa4c-ee32e820fc1d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:38.625131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ef077-a52b-4fe4-899f-9a1390135785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:38.625869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef077-a52b-4fe4-899f-9a1390135785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:38.626113 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef077-a52b-4fe4-899f-9a1390135785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:38.626389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef077-a52b-4fe4-899f-9a1390135785 admin admin] PATH_INFO: `/v3/users/5e4dd4d557084f65b5f9193e1433045d` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:38.627846 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ef077-a52b-4fe4-899f-9a1390135785 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e4dd4d557084f65b5f9193e1433045d)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:38.629825 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ef077-a52b-4fe4-899f-9a1390135785 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:38.634588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed120f-e822-445b-bda1-39584289b982 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:38.636655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed120f-e822-445b-bda1-39584289b982 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:38.668497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 940/1852] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:38.807184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-101ef077-a52b-4fe4-899f-9a1390135785 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e4dd4d557084f65b5f9193e1433045d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:38.807970 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-101ef077-a52b-4fe4-899f-9a1390135785 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e4dd4d557084f65b5f9193e1433045d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:38.838957 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-101ef077-a52b-4fe4-899f-9a1390135785 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e4dd4d557084f65b5f9193e1433045d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:38.872508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-101ef077-a52b-4fe4-899f-9a1390135785 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e4dd4d557084f65b5f9193e1433045d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:38.874667 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-101ef077-a52b-4fe4-899f-9a1390135785 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e4dd4d557084f65b5f9193e1433045d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:38.893268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 913/1853] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:34:38 2022] DELETE /identity/v3/users/5e4dd4d557084f65b5f9193e1433045d => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:38.897825 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a59661-8649-4eb2-87a4-0440b5bb7e2f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:38.898717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a59661-8649-4eb2-87a4-0440b5bb7e2f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:38.898717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a59661-8649-4eb2-87a4-0440b5bb7e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:38.933356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-44a59661-8649-4eb2-87a4-0440b5bb7e2f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:38.934980 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-44a59661-8649-4eb2-87a4-0440b5bb7e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:39.023770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 941/1854] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:34:39.038926 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ef077-a52b-4fe4-899f-9a1390135785 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:39.040428 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-101ef077-a52b-4fe4-899f-9a1390135785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:39.116428 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ef077-a52b-4fe4-899f-9a1390135785 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:39.121264 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef5b56-9324-4885-8e30-a4fed4d37c01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:39.122110 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef5b56-9324-4885-8e30-a4fed4d37c01 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:39.122650 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef5b56-9324-4885-8e30-a4fed4d37c01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:39.123604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef5b56-9324-4885-8e30-a4fed4d37c01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:39.141449 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef5b56-9324-4885-8e30-a4fed4d37c01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:39.143252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef5b56-9324-4885-8e30-a4fed4d37c01 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:39.158286 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 914/1855] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:39.520930 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a59661-8649-4eb2-87a4-0440b5bb7e2f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:39.523070 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-44a59661-8649-4eb2-87a4-0440b5bb7e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:39.553568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a59661-8649-4eb2-87a4-0440b5bb7e2f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:39.562184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede9346-59c2-4b45-9d79-4fe089407359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:39.563244 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede9346-59c2-4b45-9d79-4fe089407359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:39.563535 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede9346-59c2-4b45-9d79-4fe089407359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:39.563861 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede9346-59c2-4b45-9d79-4fe089407359 admin admin] PATH_INFO: `/v3/projects/bbd1adcd43054b50bb8a5d41a27d44bc` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:39.565069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede9346-59c2-4b45-9d79-4fe089407359 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbd1adcd43054b50bb8a5d41a27d44bc)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:39.567069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede9346-59c2-4b45-9d79-4fe089407359 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:39.629379 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dede9346-59c2-4b45-9d79-4fe089407359 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbd1adcd43054b50bb8a5d41a27d44bc', 'request_id': 'req-dede9346-59c2-4b45-9d79-4fe089407359'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:39.638582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dede9346-59c2-4b45-9d79-4fe089407359 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbd1adcd43054b50bb8a5d41a27d44bc', 'request_id': 'req-dede9346-59c2-4b45-9d79-4fe089407359'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:39.649577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dede9346-59c2-4b45-9d79-4fe089407359 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbd1adcd43054b50bb8a5d41a27d44bc', 'request_id': 'req-dede9346-59c2-4b45-9d79-4fe089407359'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:39.680293 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dede9346-59c2-4b45-9d79-4fe089407359 admin admin] The token cache is being invalidate because project bbd1adcd43054b50bb8a5d41a27d44bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:39.681340 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dede9346-59c2-4b45-9d79-4fe089407359 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:39.682623 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dede9346-59c2-4b45-9d79-4fe089407359 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:39.684398 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 942/1856] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:39 2022] DELETE /identity/v3/projects/bbd1adcd43054b50bb8a5d41a27d44bc => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:39.688939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef5b56-9324-4885-8e30-a4fed4d37c01 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:39.698954 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef5b56-9324-4885-8e30-a4fed4d37c01 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:39.764531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:39.802549 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc1501-bfce-4e09-8a0a-0ad35d1c2ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:39.803324 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc1501-bfce-4e09-8a0a-0ad35d1c2ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:39.803679 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc1501-bfce-4e09-8a0a-0ad35d1c2ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:39.804026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc1501-bfce-4e09-8a0a-0ad35d1c2ac0 admin admin] PATH_INFO: `/v3/users/b3510304f4d346b1bc7a1f0f65a0a91a` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:39.805799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc1501-bfce-4e09-8a0a-0ad35d1c2ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3510304f4d346b1bc7a1f0f65a0a91a)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:39.807939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc1501-bfce-4e09-8a0a-0ad35d1c2ac0 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:39.933070 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-dbfc1501-bfce-4e09-8a0a-0ad35d1c2ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3510304f4d346b1bc7a1f0f65a0a91a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:39.934169 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-dbfc1501-bfce-4e09-8a0a-0ad35d1c2ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3510304f4d346b1bc7a1f0f65a0a91a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:39.953291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-dbfc1501-bfce-4e09-8a0a-0ad35d1c2ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3510304f4d346b1bc7a1f0f65a0a91a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:39.976473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-dbfc1501-bfce-4e09-8a0a-0ad35d1c2ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3510304f4d346b1bc7a1f0f65a0a91a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:39.977517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-dbfc1501-bfce-4e09-8a0a-0ad35d1c2ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3510304f4d346b1bc7a1f0f65a0a91a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:39.995585 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 915/1857] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:34:39 2022] DELETE /identity/v3/users/b3510304f4d346b1bc7a1f0f65a0a91a => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:40.700430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede9346-59c2-4b45-9d79-4fe089407359 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:40.700430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-dede9346-59c2-4b45-9d79-4fe089407359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:40.721696 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede9346-59c2-4b45-9d79-4fe089407359 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:40.806485 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:40.839747 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca5dcd-0290-4a45-9522-7734d1121316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:40.839747 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca5dcd-0290-4a45-9522-7734d1121316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:40.839747 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca5dcd-0290-4a45-9522-7734d1121316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:40.840148 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca5dcd-0290-4a45-9522-7734d1121316 admin admin] PATH_INFO: `/v3/projects/6af33169e54d479c88294a7b8e1bb166` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:40.841277 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca5dcd-0290-4a45-9522-7734d1121316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6af33169e54d479c88294a7b8e1bb166)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:40.843300 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca5dcd-0290-4a45-9522-7734d1121316 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:40.942438 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-35ca5dcd-0290-4a45-9522-7734d1121316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6af33169e54d479c88294a7b8e1bb166', 'request_id': 'req-35ca5dcd-0290-4a45-9522-7734d1121316'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:40.957487 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-35ca5dcd-0290-4a45-9522-7734d1121316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6af33169e54d479c88294a7b8e1bb166', 'request_id': 'req-35ca5dcd-0290-4a45-9522-7734d1121316'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:40.970856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-35ca5dcd-0290-4a45-9522-7734d1121316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6af33169e54d479c88294a7b8e1bb166', 'request_id': 'req-35ca5dcd-0290-4a45-9522-7734d1121316'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:40.995890 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-35ca5dcd-0290-4a45-9522-7734d1121316 admin admin] The token cache is being invalidate because project 6af33169e54d479c88294a7b8e1bb166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:40.996229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-35ca5dcd-0290-4a45-9522-7734d1121316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:40.999002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-35ca5dcd-0290-4a45-9522-7734d1121316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:41.003948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 943/1858] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:40 2022] DELETE /identity/v3/projects/6af33169e54d479c88294a7b8e1bb166 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:34:41.010876 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689643cf-ae6d-48a1-a162-9516f8f5dea5 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:41.012166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689643cf-ae6d-48a1-a162-9516f8f5dea5 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:41.012780 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689643cf-ae6d-48a1-a162-9516f8f5dea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:41.073754 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-689643cf-ae6d-48a1-a162-9516f8f5dea5 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:41.075143 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-689643cf-ae6d-48a1-a162-9516f8f5dea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:41.189243 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 916/1859] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:41.197600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f24c71-9f30-4b90-9cd8-0ca1ff6c3277 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:41.198116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f24c71-9f30-4b90-9cd8-0ca1ff6c3277 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:41.198496 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f24c71-9f30-4b90-9cd8-0ca1ff6c3277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:41.221028 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-12f24c71-9f30-4b90-9cd8-0ca1ff6c3277 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:41.222398 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-12f24c71-9f30-4b90-9cd8-0ca1ff6c3277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:41.231484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 944/1860] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:41.236566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689643cf-ae6d-48a1-a162-9516f8f5dea5 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:41.246764 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689643cf-ae6d-48a1-a162-9516f8f5dea5 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:41.251901 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18518b52-fbc4-46ac-93c2-6aa8fa5f4b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:41.253002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18518b52-fbc4-46ac-93c2-6aa8fa5f4b91 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:41.253303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18518b52-fbc4-46ac-93c2-6aa8fa5f4b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:41.253562 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18518b52-fbc4-46ac-93c2-6aa8fa5f4b91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:41.254313 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18518b52-fbc4-46ac-93c2-6aa8fa5f4b91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:34:41.254688 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18518b52-fbc4-46ac-93c2-6aa8fa5f4b91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:41.256248 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18518b52-fbc4-46ac-93c2-6aa8fa5f4b91 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:41.270300 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 917/1861] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:34:41.274865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f24c71-9f30-4b90-9cd8-0ca1ff6c3277 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:41.285397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f24c71-9f30-4b90-9cd8-0ca1ff6c3277 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:41.290131 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95097c-d735-4821-b1fa-502a46a288d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:41.290761 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95097c-d735-4821-b1fa-502a46a288d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:41.291036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95097c-d735-4821-b1fa-502a46a288d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:41.291304 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95097c-d735-4821-b1fa-502a46a288d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:41.292010 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95097c-d735-4821-b1fa-502a46a288d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:41.293880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95097c-d735-4821-b1fa-502a46a288d6 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:41.323687 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 945/1862] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:34:41 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:41.327916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18518b52-fbc4-46ac-93c2-6aa8fa5f4b91 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:41.339579 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18518b52-fbc4-46ac-93c2-6aa8fa5f4b91 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:41.419147 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:41.479599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4eaa3-4032-4527-a6a7-17e050d5e2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:41.479849 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4eaa3-4032-4527-a6a7-17e050d5e2ef admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:41.480205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4eaa3-4032-4527-a6a7-17e050d5e2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:41.480578 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4eaa3-4032-4527-a6a7-17e050d5e2ef admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:41.482303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4eaa3-4032-4527-a6a7-17e050d5e2ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:41.485329 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4eaa3-4032-4527-a6a7-17e050d5e2ef admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:41.531310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 918/1863] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:34:41 2022] POST /identity/v3/users => generated 284 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:41.537145 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95097c-d735-4821-b1fa-502a46a288d6 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:41.580131 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95097c-d735-4821-b1fa-502a46a288d6 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:41.598566 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6961e7-de05-4c6b-b0e8-b89b95db77f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:41.602875 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6961e7-de05-4c6b-b0e8-b89b95db77f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:41.603744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6961e7-de05-4c6b-b0e8-b89b95db77f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:41.606056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6961e7-de05-4c6b-b0e8-b89b95db77f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:41.607884 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6961e7-de05-4c6b-b0e8-b89b95db77f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:41.610744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6961e7-de05-4c6b-b0e8-b89b95db77f4 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:41.630191 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 946/1864] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:41.635955 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4eaa3-4032-4527-a6a7-17e050d5e2ef admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:41.667920 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4eaa3-4032-4527-a6a7-17e050d5e2ef admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:41.695786 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6961e7-de05-4c6b-b0e8-b89b95db77f4 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:41.698308 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fd6961e7-de05-4c6b-b0e8-b89b95db77f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:41.700792 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6988fb0-4951-4443-bc18-edddf7508c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:41.703414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6988fb0-4951-4443-bc18-edddf7508c6b admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:41.703749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6988fb0-4951-4443-bc18-edddf7508c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:41.704203 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6988fb0-4951-4443-bc18-edddf7508c6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:41.705442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6988fb0-4951-4443-bc18-edddf7508c6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:41.708482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6988fb0-4951-4443-bc18-edddf7508c6b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:41.727853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 919/1865] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:41.740444 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6961e7-de05-4c6b-b0e8-b89b95db77f4 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:41.749479 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2f05c-e314-4c29-854a-70696e55ea4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:41.750865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2f05c-e314-4c29-854a-70696e55ea4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:41.751414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2f05c-e314-4c29-854a-70696e55ea4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:41.751908 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2f05c-e314-4c29-854a-70696e55ea4f admin admin] PATH_INFO: `/v3/users/31da4d2a391f4e879d6a2960f2bd7048` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:41.753923 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6988fb0-4951-4443-bc18-edddf7508c6b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:41.754344 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2f05c-e314-4c29-854a-70696e55ea4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=31da4d2a391f4e879d6a2960f2bd7048)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:41.758157 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2f05c-e314-4c29-854a-70696e55ea4f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:41.773794 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6988fb0-4951-4443-bc18-edddf7508c6b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:41.792332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81dcd81-3fc1-4af2-a8dc-68ce8fe0f4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:41.793043 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81dcd81-3fc1-4af2-a8dc-68ce8fe0f4fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:41.793327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81dcd81-3fc1-4af2-a8dc-68ce8fe0f4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:41.793607 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81dcd81-3fc1-4af2-a8dc-68ce8fe0f4fa admin admin] PATH_INFO: `/v3/projects/c75423e804a94078aab049b3759d35a8/users/fc9deaae39494f9584777af16dc7d75a/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:41.834902 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81dcd81-3fc1-4af2-a8dc-68ce8fe0f4fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c75423e804a94078aab049b3759d35a8, user_id=fc9deaae39494f9584777af16dc7d75a, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:41.838830 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81dcd81-3fc1-4af2-a8dc-68ce8fe0f4fa admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:41.879744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dfb2f05c-e314-4c29-854a-70696e55ea4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31da4d2a391f4e879d6a2960f2bd7048'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:41.880686 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dfb2f05c-e314-4c29-854a-70696e55ea4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31da4d2a391f4e879d6a2960f2bd7048'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:41.881473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 920/1866] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:41 2022] PUT /identity/v3/projects/c75423e804a94078aab049b3759d35a8/users/fc9deaae39494f9584777af16dc7d75a/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:41.895038 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f1fca-324b-4c5b-a62a-1cab1f462989 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:41.895420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f1fca-324b-4c5b-a62a-1cab1f462989 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:41.895773 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f1fca-324b-4c5b-a62a-1cab1f462989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:41.900574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dfb2f05c-e314-4c29-854a-70696e55ea4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31da4d2a391f4e879d6a2960f2bd7048'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:41.933062 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-be0f1fca-324b-4c5b-a62a-1cab1f462989 None None] MFA Rules not processed for user `fc9deaae39494f9584777af16dc7d75a`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:41.934629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dfb2f05c-e314-4c29-854a-70696e55ea4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31da4d2a391f4e879d6a2960f2bd7048'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:41.935117 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-be0f1fca-324b-4c5b-a62a-1cab1f462989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:41.971437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dfb2f05c-e314-4c29-854a-70696e55ea4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31da4d2a391f4e879d6a2960f2bd7048'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:41.985348 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 947/1867] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:34:41 2022] DELETE /identity/v3/users/31da4d2a391f4e879d6a2960f2bd7048 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:34:42.003249 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91021d26-e72e-4c37-976a-6d90f9068b51 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:42.012070 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91021d26-e72e-4c37-976a-6d90f9068b51 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:42.012364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91021d26-e72e-4c37-976a-6d90f9068b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:42.028231 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:42.057265 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-91021d26-e72e-4c37-976a-6d90f9068b51 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:42.058606 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-91021d26-e72e-4c37-976a-6d90f9068b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:42.109353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 921/1868] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:41 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:42.116266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f1fca-324b-4c5b-a62a-1cab1f462989 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:42.118706 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-be0f1fca-324b-4c5b-a62a-1cab1f462989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:42.231896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 948/1869] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:42.251260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91021d26-e72e-4c37-976a-6d90f9068b51 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:42.255349 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-91021d26-e72e-4c37-976a-6d90f9068b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:42.299328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f1fca-324b-4c5b-a62a-1cab1f462989 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:42.309504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489bd80-a184-4e86-a08f-92b5678bd6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:42.311394 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489bd80-a184-4e86-a08f-92b5678bd6bc admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:42.312195 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489bd80-a184-4e86-a08f-92b5678bd6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:42.312938 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489bd80-a184-4e86-a08f-92b5678bd6bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:42.314791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c489bd80-a184-4e86-a08f-92b5678bd6bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:42.321626 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c489bd80-a184-4e86-a08f-92b5678bd6bc admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:42.357694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91021d26-e72e-4c37-976a-6d90f9068b51 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:42.361816 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11b329-ef82-4f3a-ae53-c88205bda552 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:42.362493 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11b329-ef82-4f3a-ae53-c88205bda552 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:42.362780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11b329-ef82-4f3a-ae53-c88205bda552 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:42.363059 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11b329-ef82-4f3a-ae53-c88205bda552 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:42.376426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d11b329-ef82-4f3a-ae53-c88205bda552 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:42.378109 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d11b329-ef82-4f3a-ae53-c88205bda552 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:42.409016 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 949/1870] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:42.421362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 922/1871] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:34:42 2022] POST /identity/v3/projects => generated 367 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:42.424898 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11b329-ef82-4f3a-ae53-c88205bda552 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:42.448879 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11b329-ef82-4f3a-ae53-c88205bda552 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:42.637333 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d29cf6-706b-4e07-8f76-78849bab7193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:42.638686 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d29cf6-706b-4e07-8f76-78849bab7193 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:42.638971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d29cf6-706b-4e07-8f76-78849bab7193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:42.639242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d29cf6-706b-4e07-8f76-78849bab7193 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:42.639956 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d29cf6-706b-4e07-8f76-78849bab7193 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:42.646323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d29cf6-706b-4e07-8f76-78849bab7193 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:42.692604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489bd80-a184-4e86-a08f-92b5678bd6bc admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:42.694766 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 950/1872] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:34:42 2022] POST /identity/v3/users => generated 290 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:34:42.705301 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489bd80-a184-4e86-a08f-92b5678bd6bc admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:42.712926 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c2274-0a58-43ed-8ced-b3952fdd33e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:42.713754 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c2274-0a58-43ed-8ced-b3952fdd33e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:42.714191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c2274-0a58-43ed-8ced-b3952fdd33e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:42.714565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c2274-0a58-43ed-8ced-b3952fdd33e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:42.715530 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c2274-0a58-43ed-8ced-b3952fdd33e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:42.718812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c2274-0a58-43ed-8ced-b3952fdd33e1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:42.734688 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:42.756077 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d29cf6-706b-4e07-8f76-78849bab7193 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:42.757567 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 923/1873] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:42.775399 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d29cf6-706b-4e07-8f76-78849bab7193 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:42.792067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49efdcd-f58e-46b7-82fd-6f671415b910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:42.792772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49efdcd-f58e-46b7-82fd-6f671415b910 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:42.793056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49efdcd-f58e-46b7-82fd-6f671415b910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:42.793405 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49efdcd-f58e-46b7-82fd-6f671415b910 admin admin] PATH_INFO: `/v3/projects/66e138ed81c94ff2be6192823c9aab44/users/1de2040bf8444235bfc1d4845b5bb01b/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:42.827395 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49efdcd-f58e-46b7-82fd-6f671415b910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66e138ed81c94ff2be6192823c9aab44, user_id=1de2040bf8444235bfc1d4845b5bb01b, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:42.831879 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49efdcd-f58e-46b7-82fd-6f671415b910 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:42.851228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 951/1874] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:42 2022] PUT /identity/v3/projects/66e138ed81c94ff2be6192823c9aab44/users/1de2040bf8444235bfc1d4845b5bb01b/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:42.855688 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c2274-0a58-43ed-8ced-b3952fdd33e1 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:42.866141 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c2274-0a58-43ed-8ced-b3952fdd33e1 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:42.947128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39c611-f98d-4fb5-b859-5f0d82440faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:42.947750 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39c611-f98d-4fb5-b859-5f0d82440faa admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:42.948286 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39c611-f98d-4fb5-b859-5f0d82440faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:42.948576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39c611-f98d-4fb5-b859-5f0d82440faa admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:42.949382 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39c611-f98d-4fb5-b859-5f0d82440faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:42.952011 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39c611-f98d-4fb5-b859-5f0d82440faa admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:42.955342 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49efdcd-f58e-46b7-82fd-6f671415b910 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:42.957494 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f49efdcd-f58e-46b7-82fd-6f671415b910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:42.974227 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 924/1875] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:42.978382 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39c611-f98d-4fb5-b859-5f0d82440faa admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:42.986857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49efdcd-f58e-46b7-82fd-6f671415b910 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:42.994360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f8011-d1da-44e6-8a20-91e18921f8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:42.995336 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f8011-d1da-44e6-8a20-91e18921f8cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:42.995772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f8011-d1da-44e6-8a20-91e18921f8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:42.995933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f8011-d1da-44e6-8a20-91e18921f8cd admin admin] PATH_INFO: `/v3/projects/f972d0bf875c463da84a13f3ccb3eea2` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:42.997643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f8011-d1da-44e6-8a20-91e18921f8cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f972d0bf875c463da84a13f3ccb3eea2)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:43.000443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f8011-d1da-44e6-8a20-91e18921f8cd admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:43.001767 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39c611-f98d-4fb5-b859-5f0d82440faa admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:43.008091 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f5d3f-6be1-4849-a036-98286f54cd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:43.009442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f5d3f-6be1-4849-a036-98286f54cd4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:43.009830 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f5d3f-6be1-4849-a036-98286f54cd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:43.010730 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:43.011460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f5d3f-6be1-4849-a036-98286f54cd4e admin admin] PATH_INFO: `/v3/projects/66e138ed81c94ff2be6192823c9aab44/users/1de2040bf8444235bfc1d4845b5bb01b/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:43.015704 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f5d3f-6be1-4849-a036-98286f54cd4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=66e138ed81c94ff2be6192823c9aab44, user_id=1de2040bf8444235bfc1d4845b5bb01b, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:43.019999 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f5d3f-6be1-4849-a036-98286f54cd4e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:43.032939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 925/1876] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:42 2022] PUT /identity/v3/projects/66e138ed81c94ff2be6192823c9aab44/users/1de2040bf8444235bfc1d4845b5bb01b/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:43.048739 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff0314-bd75-4e24-8116-b0498f103c96 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:43.049048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff0314-bd75-4e24-8116-b0498f103c96 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:43.049470 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff0314-bd75-4e24-8116-b0498f103c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:43.106743 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-75ff0314-bd75-4e24-8116-b0498f103c96 None None] MFA Rules not processed for user `1de2040bf8444235bfc1d4845b5bb01b`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:43.109375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-75ff0314-bd75-4e24-8116-b0498f103c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:43.115668 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-104f8011-d1da-44e6-8a20-91e18921f8cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f972d0bf875c463da84a13f3ccb3eea2', 'request_id': 'req-104f8011-d1da-44e6-8a20-91e18921f8cd'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:43.133246 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-104f8011-d1da-44e6-8a20-91e18921f8cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f972d0bf875c463da84a13f3ccb3eea2', 'request_id': 'req-104f8011-d1da-44e6-8a20-91e18921f8cd'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:43.151039 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-104f8011-d1da-44e6-8a20-91e18921f8cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f972d0bf875c463da84a13f3ccb3eea2', 'request_id': 'req-104f8011-d1da-44e6-8a20-91e18921f8cd'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:43.181128 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-104f8011-d1da-44e6-8a20-91e18921f8cd admin admin] The token cache is being invalidate because project f972d0bf875c463da84a13f3ccb3eea2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:43.181436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-104f8011-d1da-44e6-8a20-91e18921f8cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:43.183338 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-104f8011-d1da-44e6-8a20-91e18921f8cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:43.184427 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 952/1877] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:42 2022] DELETE /identity/v3/projects/f972d0bf875c463da84a13f3ccb3eea2 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:43.191206 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f8011-d1da-44e6-8a20-91e18921f8cd admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:43.193645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-104f8011-d1da-44e6-8a20-91e18921f8cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:43.283495 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 926/1878] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:43 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:34:43.295187 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff0314-bd75-4e24-8116-b0498f103c96 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:43.297715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-75ff0314-bd75-4e24-8116-b0498f103c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:43.365654 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f8011-d1da-44e6-8a20-91e18921f8cd admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:43.374524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe10dc5-dcfd-4271-9832-b43614749278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:43.375754 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe10dc5-dcfd-4271-9832-b43614749278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:43.376372 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe10dc5-dcfd-4271-9832-b43614749278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:43.376846 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe10dc5-dcfd-4271-9832-b43614749278 admin admin] PATH_INFO: `/v3/users/e0c783cc45b04ea3aec1ba97980a6f67` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:43.379882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe10dc5-dcfd-4271-9832-b43614749278 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0c783cc45b04ea3aec1ba97980a6f67)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:43.390278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe10dc5-dcfd-4271-9832-b43614749278 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:43.399371 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff0314-bd75-4e24-8116-b0498f103c96 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:43.406748 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3088b-abd9-414c-a63f-6fa8c753a2f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:43.407407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3088b-abd9-414c-a63f-6fa8c753a2f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:43.407749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3088b-abd9-414c-a63f-6fa8c753a2f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:43.408033 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3088b-abd9-414c-a63f-6fa8c753a2f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:43.437366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3088b-abd9-414c-a63f-6fa8c753a2f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:43.439092 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3088b-abd9-414c-a63f-6fa8c753a2f5 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:43.465981 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 927/1879] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:43 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:43.496169 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8fe10dc5-dcfd-4271-9832-b43614749278 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0c783cc45b04ea3aec1ba97980a6f67'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:43.498729 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8fe10dc5-dcfd-4271-9832-b43614749278 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0c783cc45b04ea3aec1ba97980a6f67'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:43.522164 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8fe10dc5-dcfd-4271-9832-b43614749278 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0c783cc45b04ea3aec1ba97980a6f67'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:43.551341 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8fe10dc5-dcfd-4271-9832-b43614749278 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0c783cc45b04ea3aec1ba97980a6f67'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:43.572136 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8fe10dc5-dcfd-4271-9832-b43614749278 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0c783cc45b04ea3aec1ba97980a6f67'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:43.596721 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 953/1880] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:34:43 2022] DELETE /identity/v3/users/e0c783cc45b04ea3aec1ba97980a6f67 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:43.656717 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3088b-abd9-414c-a63f-6fa8c753a2f5 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:43.673620 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3088b-abd9-414c-a63f-6fa8c753a2f5 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:43.710227 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:43.874227 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c094a43-da83-430a-8c36-74e04f01155e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:43.875013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c094a43-da83-430a-8c36-74e04f01155e service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:43.875354 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c094a43-da83-430a-8c36-74e04f01155e service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:43.875632 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c094a43-da83-430a-8c36-74e04f01155e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:43.877746 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3c094a43-da83-430a-8c36-74e04f01155e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:43.976840 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c094a43-da83-430a-8c36-74e04f01155e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:43.978598 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c094a43-da83-430a-8c36-74e04f01155e service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:43.991051 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 928/1881] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:43 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:44.905128 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe10dc5-dcfd-4271-9832-b43614749278 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:44.908140 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8fe10dc5-dcfd-4271-9832-b43614749278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:44.923604 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:45.001363 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe10dc5-dcfd-4271-9832-b43614749278 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:45.006722 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b12ae-9ac9-4ba4-9f03-a194ce48b556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:45.007637 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b12ae-9ac9-4ba4-9f03-a194ce48b556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:45.007926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b12ae-9ac9-4ba4-9f03-a194ce48b556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:45.008201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b12ae-9ac9-4ba4-9f03-a194ce48b556 admin admin] PATH_INFO: `/v3/projects/ff6020c9a74549ff9240edbc6964e352` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:45.009451 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b12ae-9ac9-4ba4-9f03-a194ce48b556 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff6020c9a74549ff9240edbc6964e352)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:45.012149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b12ae-9ac9-4ba4-9f03-a194ce48b556 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:45.075716 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1b7b12ae-9ac9-4ba4-9f03-a194ce48b556 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff6020c9a74549ff9240edbc6964e352', 'request_id': 'req-1b7b12ae-9ac9-4ba4-9f03-a194ce48b556'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:45.083005 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1b7b12ae-9ac9-4ba4-9f03-a194ce48b556 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff6020c9a74549ff9240edbc6964e352', 'request_id': 'req-1b7b12ae-9ac9-4ba4-9f03-a194ce48b556'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:45.102617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1b7b12ae-9ac9-4ba4-9f03-a194ce48b556 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff6020c9a74549ff9240edbc6964e352', 'request_id': 'req-1b7b12ae-9ac9-4ba4-9f03-a194ce48b556'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:45.119357 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1b7b12ae-9ac9-4ba4-9f03-a194ce48b556 admin admin] The token cache is being invalidate because project ff6020c9a74549ff9240edbc6964e352 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:45.120002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1b7b12ae-9ac9-4ba4-9f03-a194ce48b556 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:45.121055 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1b7b12ae-9ac9-4ba4-9f03-a194ce48b556 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:45.124098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 954/1882] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:44 2022] DELETE /identity/v3/projects/ff6020c9a74549ff9240edbc6964e352 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:34:45.128610 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15db06-ba54-44dd-88ba-767f9930bc2a None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:45.129131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15db06-ba54-44dd-88ba-767f9930bc2a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:45.129209 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15db06-ba54-44dd-88ba-767f9930bc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:45.152291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-af15db06-ba54-44dd-88ba-767f9930bc2a None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:45.153626 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-af15db06-ba54-44dd-88ba-767f9930bc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:45.252255 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 929/1883] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:45.264173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716fd60-cc91-4e34-8f38-d79db8ab7800 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:45.264500 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716fd60-cc91-4e34-8f38-d79db8ab7800 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:45.264854 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716fd60-cc91-4e34-8f38-d79db8ab7800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:45.292445 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-5716fd60-cc91-4e34-8f38-d79db8ab7800 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:45.293658 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5716fd60-cc91-4e34-8f38-d79db8ab7800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:45.307936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 955/1884] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:45.313058 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15db06-ba54-44dd-88ba-767f9930bc2a None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:45.322960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15db06-ba54-44dd-88ba-767f9930bc2a None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:45.328400 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7a919-c2b1-410c-b7c9-1aebdeafd5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:45.329116 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7a919-c2b1-410c-b7c9-1aebdeafd5fa admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:45.329486 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7a919-c2b1-410c-b7c9-1aebdeafd5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:45.329830 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7a919-c2b1-410c-b7c9-1aebdeafd5fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:45.330952 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f7a919-c2b1-410c-b7c9-1aebdeafd5fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:34:45.331365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f7a919-c2b1-410c-b7c9-1aebdeafd5fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:45.333200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f7a919-c2b1-410c-b7c9-1aebdeafd5fa admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:45.359878 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 930/1885] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:34:45.364327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716fd60-cc91-4e34-8f38-d79db8ab7800 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:45.379909 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716fd60-cc91-4e34-8f38-d79db8ab7800 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:45.386440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b87f3b-6215-43b8-ba3b-25c892433d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:45.387090 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b87f3b-6215-43b8-ba3b-25c892433d4d admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:45.387374 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b87f3b-6215-43b8-ba3b-25c892433d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:45.387649 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b87f3b-6215-43b8-ba3b-25c892433d4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:45.388874 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b87f3b-6215-43b8-ba3b-25c892433d4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:45.391014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b87f3b-6215-43b8-ba3b-25c892433d4d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:45.437707 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 956/1886] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:34:45 2022] POST /identity/v3/projects => generated 407 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:45.440969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7a919-c2b1-410c-b7c9-1aebdeafd5fa admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:45.457862 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7a919-c2b1-410c-b7c9-1aebdeafd5fa admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:45.516653 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:45.529640 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efa58f-82f0-4527-a608-6e9f918461ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:45.529640 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efa58f-82f0-4527-a608-6e9f918461ea admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:45.529640 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efa58f-82f0-4527-a608-6e9f918461ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:45.530211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efa58f-82f0-4527-a608-6e9f918461ea admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:45.530883 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06efa58f-82f0-4527-a608-6e9f918461ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:45.532747 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06efa58f-82f0-4527-a608-6e9f918461ea admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:45.568429 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 931/1887] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:34:45 2022] POST /identity/v3/users => generated 304 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:45.573773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b87f3b-6215-43b8-ba3b-25c892433d4d admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:45.589523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b87f3b-6215-43b8-ba3b-25c892433d4d admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:45.600522 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156139f-cd29-45c8-920f-a70a0d0b2e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:45.601183 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156139f-cd29-45c8-920f-a70a0d0b2e8b admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:45.602939 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156139f-cd29-45c8-920f-a70a0d0b2e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:45.603631 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156139f-cd29-45c8-920f-a70a0d0b2e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:45.604388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156139f-cd29-45c8-920f-a70a0d0b2e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:45.606611 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156139f-cd29-45c8-920f-a70a0d0b2e8b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:45.627319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 957/1888] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:45.627826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efa58f-82f0-4527-a608-6e9f918461ea admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:45.648653 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efa58f-82f0-4527-a608-6e9f918461ea admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:45.668940 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d60096-3714-4123-8d46-35fa9f9097bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:45.671554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d60096-3714-4123-8d46-35fa9f9097bf admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:45.671554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d60096-3714-4123-8d46-35fa9f9097bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:45.671554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d60096-3714-4123-8d46-35fa9f9097bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:45.672068 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d60096-3714-4123-8d46-35fa9f9097bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:45.676687 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d60096-3714-4123-8d46-35fa9f9097bf admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:45.709248 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 932/1889] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:45.710452 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156139f-cd29-45c8-920f-a70a0d0b2e8b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:45.725227 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156139f-cd29-45c8-920f-a70a0d0b2e8b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:45.758050 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05393127-2528-440a-bfba-38d80dce51c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:45.761350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05393127-2528-440a-bfba-38d80dce51c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:45.762064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05393127-2528-440a-bfba-38d80dce51c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:45.763267 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05393127-2528-440a-bfba-38d80dce51c1 admin admin] PATH_INFO: `/v3/projects/b21f0dea9004491eaae324a40b339670/users/319bbc3f15a64d808fcb4b7c35d393b2/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:45.830066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05393127-2528-440a-bfba-38d80dce51c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21f0dea9004491eaae324a40b339670, user_id=319bbc3f15a64d808fcb4b7c35d393b2, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:45.833982 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05393127-2528-440a-bfba-38d80dce51c1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:45.879581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 958/1890] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:45 2022] PUT /identity/v3/projects/b21f0dea9004491eaae324a40b339670/users/319bbc3f15a64d808fcb4b7c35d393b2/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:45.892269 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70168f-3c19-4280-9a8f-ca66d61d676b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:45.892571 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70168f-3c19-4280-9a8f-ca66d61d676b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:45.892850 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70168f-3c19-4280-9a8f-ca66d61d676b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:45.965226 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-5e70168f-3c19-4280-9a8f-ca66d61d676b None None] MFA Rules not processed for user `319bbc3f15a64d808fcb4b7c35d393b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:45.966707 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5e70168f-3c19-4280-9a8f-ca66d61d676b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:46.240907 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 933/1891] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:45 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:46.243296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05393127-2528-440a-bfba-38d80dce51c1 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:46.341111 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:46.404401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05393127-2528-440a-bfba-38d80dce51c1 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:46.412678 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dc61e-a907-44b9-ae61-1e199cba0e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:46.414091 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dc61e-a907-44b9-ae61-1e199cba0e5d admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:46.415147 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dc61e-a907-44b9-ae61-1e199cba0e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:46.416304 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dc61e-a907-44b9-ae61-1e199cba0e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:46.417533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dc61e-a907-44b9-ae61-1e199cba0e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:46.420710 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dc61e-a907-44b9-ae61-1e199cba0e5d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:46.464276 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70168f-3c19-4280-9a8f-ca66d61d676b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:46.472404 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 959/1892] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:34:46 2022] POST /identity/v3/projects => generated 407 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:46.481077 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70168f-3c19-4280-9a8f-ca66d61d676b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:46.637831 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245349b-80b8-4a1a-8ee4-143e740d4b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:46.638670 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245349b-80b8-4a1a-8ee4-143e740d4b79 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:46.639000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245349b-80b8-4a1a-8ee4-143e740d4b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:46.639311 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245349b-80b8-4a1a-8ee4-143e740d4b79 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:46.640002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245349b-80b8-4a1a-8ee4-143e740d4b79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:46.646226 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245349b-80b8-4a1a-8ee4-143e740d4b79 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:46.688006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 934/1893] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:34:46 2022] POST /identity/v3/users => generated 310 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:46.692008 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dc61e-a907-44b9-ae61-1e199cba0e5d admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:46.711665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dc61e-a907-44b9-ae61-1e199cba0e5d admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:46.718283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6555fe5-7efe-429c-856e-b0726be423e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:46.719070 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6555fe5-7efe-429c-856e-b0726be423e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:46.719204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6555fe5-7efe-429c-856e-b0726be423e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:46.719480 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6555fe5-7efe-429c-856e-b0726be423e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:46.720186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6555fe5-7efe-429c-856e-b0726be423e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:46.722373 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6555fe5-7efe-429c-856e-b0726be423e9 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:46.742002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 960/1894] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:46.743135 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245349b-80b8-4a1a-8ee4-143e740d4b79 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:46.760642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245349b-80b8-4a1a-8ee4-143e740d4b79 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:46.778704 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dd466-d844-441f-be00-43c87ad74837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:46.781564 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dd466-d844-441f-be00-43c87ad74837 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:46.781564 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dd466-d844-441f-be00-43c87ad74837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:46.781564 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dd466-d844-441f-be00-43c87ad74837 admin admin] PATH_INFO: `/v3/projects/740a8982fd7f48768193372fa069ea35/users/7cf52d5f3f164a61b7b57ac4415408d4/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:46.805162 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72dd466-d844-441f-be00-43c87ad74837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=740a8982fd7f48768193372fa069ea35, user_id=7cf52d5f3f164a61b7b57ac4415408d4, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:46.809820 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72dd466-d844-441f-be00-43c87ad74837 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:46.840452 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 935/1895] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:46 2022] PUT /identity/v3/projects/740a8982fd7f48768193372fa069ea35/users/7cf52d5f3f164a61b7b57ac4415408d4/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:46.847807 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6555fe5-7efe-429c-856e-b0726be423e9 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:46.875633 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6555fe5-7efe-429c-856e-b0726be423e9 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:47.069852 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c091100-d3ef-4513-abfc-b2d673ee7079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:47.072704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c091100-d3ef-4513-abfc-b2d673ee7079 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:47.073000 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c091100-d3ef-4513-abfc-b2d673ee7079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:47.073316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c091100-d3ef-4513-abfc-b2d673ee7079 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:47.074134 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c091100-d3ef-4513-abfc-b2d673ee7079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:47.083283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c091100-d3ef-4513-abfc-b2d673ee7079 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:47.111138 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 961/1896] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:47.116139 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dd466-d844-441f-be00-43c87ad74837 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:47.134108 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dd466-d844-441f-be00-43c87ad74837 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:47.143991 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2d382-b0ad-4419-a890-957b98b1fecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:47.146140 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2d382-b0ad-4419-a890-957b98b1fecb admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:47.146140 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2d382-b0ad-4419-a890-957b98b1fecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:47.146482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2d382-b0ad-4419-a890-957b98b1fecb admin admin] PATH_INFO: `/v3/projects/740a8982fd7f48768193372fa069ea35/users/7cf52d5f3f164a61b7b57ac4415408d4/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:47.149061 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2d382-b0ad-4419-a890-957b98b1fecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=740a8982fd7f48768193372fa069ea35, user_id=7cf52d5f3f164a61b7b57ac4415408d4, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:47.152294 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2d382-b0ad-4419-a890-957b98b1fecb admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:47.187384 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 936/1897] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:47 2022] PUT /identity/v3/projects/740a8982fd7f48768193372fa069ea35/users/7cf52d5f3f164a61b7b57ac4415408d4/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:47.205939 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad4f64-892f-4876-9581-2d4f7ddd5d40 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:47.206211 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad4f64-892f-4876-9581-2d4f7ddd5d40 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:47.207036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad4f64-892f-4876-9581-2d4f7ddd5d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:47.293422 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-7cad4f64-892f-4876-9581-2d4f7ddd5d40 None None] MFA Rules not processed for user `7cf52d5f3f164a61b7b57ac4415408d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:47.295936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7cad4f64-892f-4876-9581-2d4f7ddd5d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:47.451287 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 962/1898] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:47 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:34:47.518304 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2d382-b0ad-4419-a890-957b98b1fecb admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:47.521251 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-39d2d382-b0ad-4419-a890-957b98b1fecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:47.555040 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:47.721448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2d382-b0ad-4419-a890-957b98b1fecb admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:47.735811 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639f9ef-8ff4-4aad-87a5-f268ee0cc0f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:47.739182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639f9ef-8ff4-4aad-87a5-f268ee0cc0f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:47.740536 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639f9ef-8ff4-4aad-87a5-f268ee0cc0f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:47.742901 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639f9ef-8ff4-4aad-87a5-f268ee0cc0f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:47.762668 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639f9ef-8ff4-4aad-87a5-f268ee0cc0f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:47.765755 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639f9ef-8ff4-4aad-87a5-f268ee0cc0f0 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:47.779083 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 937/1899] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:47 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:48.219773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad4f64-892f-4876-9581-2d4f7ddd5d40 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:48.231042 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad4f64-892f-4876-9581-2d4f7ddd5d40 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:48.238221 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceaf22b-8fbd-41c0-a700-631c84aaadea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:48.239948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceaf22b-8fbd-41c0-a700-631c84aaadea service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:48.240264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceaf22b-8fbd-41c0-a700-631c84aaadea service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:48.240642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceaf22b-8fbd-41c0-a700-631c84aaadea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:48.302144 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:48.350909 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceaf22b-8fbd-41c0-a700-631c84aaadea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:48.352532 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceaf22b-8fbd-41c0-a700-631c84aaadea service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:48.366605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 963/1900] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:48 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:50.714855 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639f9ef-8ff4-4aad-87a5-f268ee0cc0f0 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:50.716667 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3639f9ef-8ff4-4aad-87a5-f268ee0cc0f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:50.804103 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639f9ef-8ff4-4aad-87a5-f268ee0cc0f0 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:50.810050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd5aed-e20c-4103-b5c1-170d59645f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:50.812520 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd5aed-e20c-4103-b5c1-170d59645f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:50.812868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd5aed-e20c-4103-b5c1-170d59645f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:50.813208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd5aed-e20c-4103-b5c1-170d59645f36 admin admin] PATH_INFO: `/v3/users/fc9deaae39494f9584777af16dc7d75a` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:50.814956 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd5aed-e20c-4103-b5c1-170d59645f36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc9deaae39494f9584777af16dc7d75a)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:50.817005 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd5aed-e20c-4103-b5c1-170d59645f36 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:50.893827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9ecd5aed-e20c-4103-b5c1-170d59645f36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc9deaae39494f9584777af16dc7d75a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:50.894809 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9ecd5aed-e20c-4103-b5c1-170d59645f36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc9deaae39494f9584777af16dc7d75a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:50.923463 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9ecd5aed-e20c-4103-b5c1-170d59645f36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc9deaae39494f9584777af16dc7d75a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:50.947092 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9ecd5aed-e20c-4103-b5c1-170d59645f36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc9deaae39494f9584777af16dc7d75a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:50.948134 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9ecd5aed-e20c-4103-b5c1-170d59645f36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc9deaae39494f9584777af16dc7d75a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:50.969959 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 938/1901] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:34:50 2022] DELETE /identity/v3/users/fc9deaae39494f9584777af16dc7d75a => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:34:50.972919 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f39e6-a721-4e6c-88f0-53646f6b234b None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:50.973438 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f39e6-a721-4e6c-88f0-53646f6b234b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:50.973530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f39e6-a721-4e6c-88f0-53646f6b234b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:51.003077 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-125f39e6-a721-4e6c-88f0-53646f6b234b None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:51.005042 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-125f39e6-a721-4e6c-88f0-53646f6b234b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:51.129495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 964/1902] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:51.152046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd5aed-e20c-4103-b5c1-170d59645f36 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:51.153482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9ecd5aed-e20c-4103-b5c1-170d59645f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:51.204573 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:51.231287 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd5aed-e20c-4103-b5c1-170d59645f36 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:51.237720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f2498-54e9-4e7e-aca3-5dc0e65d7045 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:51.238862 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f2498-54e9-4e7e-aca3-5dc0e65d7045 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:51.238862 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f2498-54e9-4e7e-aca3-5dc0e65d7045 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:51.239062 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f2498-54e9-4e7e-aca3-5dc0e65d7045 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:51.253973 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f2498-54e9-4e7e-aca3-5dc0e65d7045 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:51.255633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f2498-54e9-4e7e-aca3-5dc0e65d7045 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:51.267440 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 939/1903] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:51.916805 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f39e6-a721-4e6c-88f0-53646f6b234b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:51.918659 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-125f39e6-a721-4e6c-88f0-53646f6b234b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:51.944090 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f39e6-a721-4e6c-88f0-53646f6b234b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:51.973108 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852cf73-3d40-4712-a7f7-1ee80ed91b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:51.974115 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852cf73-3d40-4712-a7f7-1ee80ed91b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:51.974115 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852cf73-3d40-4712-a7f7-1ee80ed91b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:51.974532 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852cf73-3d40-4712-a7f7-1ee80ed91b95 admin admin] PATH_INFO: `/v3/projects/c75423e804a94078aab049b3759d35a8` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:51.976053 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852cf73-3d40-4712-a7f7-1ee80ed91b95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c75423e804a94078aab049b3759d35a8)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:51.978282 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852cf73-3d40-4712-a7f7-1ee80ed91b95 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:52.063699 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6852cf73-3d40-4712-a7f7-1ee80ed91b95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c75423e804a94078aab049b3759d35a8', 'request_id': 'req-6852cf73-3d40-4712-a7f7-1ee80ed91b95'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:52.072595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6852cf73-3d40-4712-a7f7-1ee80ed91b95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c75423e804a94078aab049b3759d35a8', 'request_id': 'req-6852cf73-3d40-4712-a7f7-1ee80ed91b95'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:52.095206 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6852cf73-3d40-4712-a7f7-1ee80ed91b95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c75423e804a94078aab049b3759d35a8', 'request_id': 'req-6852cf73-3d40-4712-a7f7-1ee80ed91b95'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:52.114895 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6852cf73-3d40-4712-a7f7-1ee80ed91b95 admin admin] The token cache is being invalidate because project c75423e804a94078aab049b3759d35a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:52.115245 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6852cf73-3d40-4712-a7f7-1ee80ed91b95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:52.115936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6852cf73-3d40-4712-a7f7-1ee80ed91b95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:52.121194 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 965/1904] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:51 2022] DELETE /identity/v3/projects/c75423e804a94078aab049b3759d35a8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:34:52.122358 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f2498-54e9-4e7e-aca3-5dc0e65d7045 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:52.136259 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f2498-54e9-4e7e-aca3-5dc0e65d7045 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:52.215506 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d893f1b-8298-4dcd-9294-0813a0c2c787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:52.216507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d893f1b-8298-4dcd-9294-0813a0c2c787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:52.216795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d893f1b-8298-4dcd-9294-0813a0c2c787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:52.217247 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d893f1b-8298-4dcd-9294-0813a0c2c787 admin admin] PATH_INFO: `/v3/users/1de2040bf8444235bfc1d4845b5bb01b` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:52.218935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d893f1b-8298-4dcd-9294-0813a0c2c787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1de2040bf8444235bfc1d4845b5bb01b)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:52.220763 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d893f1b-8298-4dcd-9294-0813a0c2c787 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:52.291173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5d893f1b-8298-4dcd-9294-0813a0c2c787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1de2040bf8444235bfc1d4845b5bb01b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:52.292691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5d893f1b-8298-4dcd-9294-0813a0c2c787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1de2040bf8444235bfc1d4845b5bb01b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:52.305603 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5d893f1b-8298-4dcd-9294-0813a0c2c787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1de2040bf8444235bfc1d4845b5bb01b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:52.331375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5d893f1b-8298-4dcd-9294-0813a0c2c787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1de2040bf8444235bfc1d4845b5bb01b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:52.332544 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5d893f1b-8298-4dcd-9294-0813a0c2c787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1de2040bf8444235bfc1d4845b5bb01b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:52.358806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 940/1905] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:34:52 2022] DELETE /identity/v3/users/1de2040bf8444235bfc1d4845b5bb01b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:52.859377 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852cf73-3d40-4712-a7f7-1ee80ed91b95 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:52.863838 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6852cf73-3d40-4712-a7f7-1ee80ed91b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:52.879772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852cf73-3d40-4712-a7f7-1ee80ed91b95 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:52.925872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d893f1b-8298-4dcd-9294-0813a0c2c787 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:52.928127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5d893f1b-8298-4dcd-9294-0813a0c2c787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:52.958871 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:52.971390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8413c-fde9-4b6e-80fb-4654fbea6b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:52.972080 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8413c-fde9-4b6e-80fb-4654fbea6b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:52.972376 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8413c-fde9-4b6e-80fb-4654fbea6b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:52.972660 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8413c-fde9-4b6e-80fb-4654fbea6b9b admin admin] PATH_INFO: `/v3/users/e7e5c59af98b45dab5fff8fdc83d6cb2` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:52.974229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e8413c-fde9-4b6e-80fb-4654fbea6b9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7e5c59af98b45dab5fff8fdc83d6cb2)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:52.976825 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e8413c-fde9-4b6e-80fb-4654fbea6b9b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:53.054302 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d4e8413c-fde9-4b6e-80fb-4654fbea6b9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7e5c59af98b45dab5fff8fdc83d6cb2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.056243 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d4e8413c-fde9-4b6e-80fb-4654fbea6b9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7e5c59af98b45dab5fff8fdc83d6cb2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.071613 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:53.077956 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d4e8413c-fde9-4b6e-80fb-4654fbea6b9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7e5c59af98b45dab5fff8fdc83d6cb2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.085080 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d893f1b-8298-4dcd-9294-0813a0c2c787 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:53.092320 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf201a2-acfe-410a-8b24-7f6cd80e94d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:53.093122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf201a2-acfe-410a-8b24-7f6cd80e94d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:53.093468 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf201a2-acfe-410a-8b24-7f6cd80e94d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:53.093741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d4e8413c-fde9-4b6e-80fb-4654fbea6b9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7e5c59af98b45dab5fff8fdc83d6cb2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.093855 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf201a2-acfe-410a-8b24-7f6cd80e94d4 admin admin] PATH_INFO: `/v3/projects/66e138ed81c94ff2be6192823c9aab44` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:53.095300 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf201a2-acfe-410a-8b24-7f6cd80e94d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66e138ed81c94ff2be6192823c9aab44)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:53.098464 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf201a2-acfe-410a-8b24-7f6cd80e94d4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:53.105210 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d4e8413c-fde9-4b6e-80fb-4654fbea6b9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7e5c59af98b45dab5fff8fdc83d6cb2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.114304 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 966/1906] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:34:52 2022] DELETE /identity/v3/users/e7e5c59af98b45dab5fff8fdc83d6cb2 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:53.194156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4bf201a2-acfe-410a-8b24-7f6cd80e94d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66e138ed81c94ff2be6192823c9aab44', 'request_id': 'req-4bf201a2-acfe-410a-8b24-7f6cd80e94d4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.204736 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4bf201a2-acfe-410a-8b24-7f6cd80e94d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66e138ed81c94ff2be6192823c9aab44', 'request_id': 'req-4bf201a2-acfe-410a-8b24-7f6cd80e94d4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.215645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4bf201a2-acfe-410a-8b24-7f6cd80e94d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66e138ed81c94ff2be6192823c9aab44', 'request_id': 'req-4bf201a2-acfe-410a-8b24-7f6cd80e94d4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.233684 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4bf201a2-acfe-410a-8b24-7f6cd80e94d4 admin admin] The token cache is being invalidate because project 66e138ed81c94ff2be6192823c9aab44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:53.234347 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4bf201a2-acfe-410a-8b24-7f6cd80e94d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.235280 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4bf201a2-acfe-410a-8b24-7f6cd80e94d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.237239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 941/1907] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:52 2022] DELETE /identity/v3/projects/66e138ed81c94ff2be6192823c9aab44 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:53.242801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146fa992-e8dc-4aff-81a4-76168e4cd831 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:53.243414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146fa992-e8dc-4aff-81a4-76168e4cd831 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:53.243922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146fa992-e8dc-4aff-81a4-76168e4cd831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:53.273039 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-146fa992-e8dc-4aff-81a4-76168e4cd831 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:53.274688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-146fa992-e8dc-4aff-81a4-76168e4cd831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:53.376376 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf201a2-acfe-410a-8b24-7f6cd80e94d4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:53.379096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4bf201a2-acfe-410a-8b24-7f6cd80e94d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:53.443691 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 967/1908] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:53.445243 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de7ac6-4a8b-4eca-9d15-4814128a935d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:53.445538 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de7ac6-4a8b-4eca-9d15-4814128a935d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:53.445817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de7ac6-4a8b-4eca-9d15-4814128a935d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:53.492019 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-64de7ac6-4a8b-4eca-9d15-4814128a935d None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:53.493489 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-64de7ac6-4a8b-4eca-9d15-4814128a935d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:53.498737 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf201a2-acfe-410a-8b24-7f6cd80e94d4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:53.503170 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 968/1909] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:53.508409 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024cecd-109e-4177-80df-ae4630230200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:53.508615 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de7ac6-4a8b-4eca-9d15-4814128a935d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:53.509295 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024cecd-109e-4177-80df-ae4630230200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:53.509484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024cecd-109e-4177-80df-ae4630230200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:53.509759 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024cecd-109e-4177-80df-ae4630230200 admin admin] PATH_INFO: `/v3/projects/43b5a672819a44f5bb5597a2d407bcda` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:53.511151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024cecd-109e-4177-80df-ae4630230200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43b5a672819a44f5bb5597a2d407bcda)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:53.513196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024cecd-109e-4177-80df-ae4630230200 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:53.520733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de7ac6-4a8b-4eca-9d15-4814128a935d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:53.526853 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c61b9-9f58-484e-905d-8cab442aca52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:53.527628 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c61b9-9f58-484e-905d-8cab442aca52 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:53.527953 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c61b9-9f58-484e-905d-8cab442aca52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:53.528217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c61b9-9f58-484e-905d-8cab442aca52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:53.529002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c61b9-9f58-484e-905d-8cab442aca52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:34:53.529304 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c61b9-9f58-484e-905d-8cab442aca52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:53.531090 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c61b9-9f58-484e-905d-8cab442aca52 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:53.547655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 969/1910] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:34:53.553600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c61b9-9f58-484e-905d-8cab442aca52 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:53.571176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c61b9-9f58-484e-905d-8cab442aca52 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:53.573148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3024cecd-109e-4177-80df-ae4630230200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43b5a672819a44f5bb5597a2d407bcda', 'request_id': 'req-3024cecd-109e-4177-80df-ae4630230200'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.581332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3024cecd-109e-4177-80df-ae4630230200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43b5a672819a44f5bb5597a2d407bcda', 'request_id': 'req-3024cecd-109e-4177-80df-ae4630230200'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.589993 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3024cecd-109e-4177-80df-ae4630230200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43b5a672819a44f5bb5597a2d407bcda', 'request_id': 'req-3024cecd-109e-4177-80df-ae4630230200'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.607068 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3024cecd-109e-4177-80df-ae4630230200 admin admin] The token cache is being invalidate because project 43b5a672819a44f5bb5597a2d407bcda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:53.607651 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3024cecd-109e-4177-80df-ae4630230200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.608412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3024cecd-109e-4177-80df-ae4630230200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.610353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 942/1911] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:53 2022] DELETE /identity/v3/projects/43b5a672819a44f5bb5597a2d407bcda => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:53.615357 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024cecd-109e-4177-80df-ae4630230200 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:53.616860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3024cecd-109e-4177-80df-ae4630230200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:53.641757 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:53.666836 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:53.682430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf98af3-4c0d-47b5-9193-dc00bd733508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:53.683452 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf98af3-4c0d-47b5-9193-dc00bd733508 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:53.684643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf98af3-4c0d-47b5-9193-dc00bd733508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:53.685151 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf98af3-4c0d-47b5-9193-dc00bd733508 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:53.686217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf98af3-4c0d-47b5-9193-dc00bd733508 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:53.691071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf98af3-4c0d-47b5-9193-dc00bd733508 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:53.719189 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024cecd-109e-4177-80df-ae4630230200 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:53.724660 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c89589-29be-449b-b89f-56d9015d76ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:53.725445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c89589-29be-449b-b89f-56d9015d76ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:53.725749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c89589-29be-449b-b89f-56d9015d76ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:53.726303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c89589-29be-449b-b89f-56d9015d76ce admin admin] PATH_INFO: `/v3/users/8daf22a6b42844b3a8db7affe1016a84` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:53.727690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c89589-29be-449b-b89f-56d9015d76ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=8daf22a6b42844b3a8db7affe1016a84)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:53.729672 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c89589-29be-449b-b89f-56d9015d76ce admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:53.747072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 970/1912] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:34:53 2022] POST /identity/v3/projects => generated 373 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:53.752323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf98af3-4c0d-47b5-9193-dc00bd733508 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:53.756910 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-cdf98af3-4c0d-47b5-9193-dc00bd733508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:53.817188 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-16c89589-29be-449b-b89f-56d9015d76ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8daf22a6b42844b3a8db7affe1016a84'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.818540 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-16c89589-29be-449b-b89f-56d9015d76ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8daf22a6b42844b3a8db7affe1016a84'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.839134 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-16c89589-29be-449b-b89f-56d9015d76ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8daf22a6b42844b3a8db7affe1016a84'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.856912 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-16c89589-29be-449b-b89f-56d9015d76ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8daf22a6b42844b3a8db7affe1016a84'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.857719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-16c89589-29be-449b-b89f-56d9015d76ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8daf22a6b42844b3a8db7affe1016a84'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.865488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf98af3-4c0d-47b5-9193-dc00bd733508 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:53.871458 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b6a83-1622-4768-ba18-9e8d93b22e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:53.872228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b6a83-1622-4768-ba18-9e8d93b22e02 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:53.873141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b6a83-1622-4768-ba18-9e8d93b22e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:53.873513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b6a83-1622-4768-ba18-9e8d93b22e02 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:53.874370 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b6a83-1622-4768-ba18-9e8d93b22e02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:53.876408 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b6a83-1622-4768-ba18-9e8d93b22e02 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:53.878419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 943/1913] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:34:53 2022] DELETE /identity/v3/users/8daf22a6b42844b3a8db7affe1016a84 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:34:53.920520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 971/1914] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:34:53 2022] POST /identity/v3/users => generated 287 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:53.925593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c89589-29be-449b-b89f-56d9015d76ce admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:53.928581 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-16c89589-29be-449b-b89f-56d9015d76ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:54.034367 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c89589-29be-449b-b89f-56d9015d76ce admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:54.040997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67656ad6-8bff-49e3-a2a2-74047b24a310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:54.042005 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67656ad6-8bff-49e3-a2a2-74047b24a310 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:54.042304 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67656ad6-8bff-49e3-a2a2-74047b24a310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:54.042583 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67656ad6-8bff-49e3-a2a2-74047b24a310 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:54.043300 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67656ad6-8bff-49e3-a2a2-74047b24a310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:54.044970 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67656ad6-8bff-49e3-a2a2-74047b24a310 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:54.057029 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 944/1915] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:53 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:54.063663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b6a83-1622-4768-ba18-9e8d93b22e02 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:54.082757 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b6a83-1622-4768-ba18-9e8d93b22e02 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:54.094793 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac0639-1faa-42b4-ae50-75e2c95e35aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:54.096075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac0639-1faa-42b4-ae50-75e2c95e35aa admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:54.096749 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac0639-1faa-42b4-ae50-75e2c95e35aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:54.097362 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac0639-1faa-42b4-ae50-75e2c95e35aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:54.098740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac0639-1faa-42b4-ae50-75e2c95e35aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:54.102339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac0639-1faa-42b4-ae50-75e2c95e35aa admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:54.126423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 972/1916] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:54.131684 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67656ad6-8bff-49e3-a2a2-74047b24a310 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:54.151110 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67656ad6-8bff-49e3-a2a2-74047b24a310 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:54.159553 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238039a-2553-494e-8776-719a40f25520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:54.161028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238039a-2553-494e-8776-719a40f25520 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:54.161328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238039a-2553-494e-8776-719a40f25520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:54.161613 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238039a-2553-494e-8776-719a40f25520 admin admin] PATH_INFO: `/v3/projects/8454f4c13e654ee58e6ff6265b7b85f9/users/c6591fd1ff63406c86b847870570e56c/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:54.171450 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac0639-1faa-42b4-ae50-75e2c95e35aa admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:54.173995 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-38ac0639-1faa-42b4-ae50-75e2c95e35aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:54.182680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238039a-2553-494e-8776-719a40f25520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8454f4c13e654ee58e6ff6265b7b85f9, user_id=c6591fd1ff63406c86b847870570e56c, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:54.189119 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238039a-2553-494e-8776-719a40f25520 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:54.195499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac0639-1faa-42b4-ae50-75e2c95e35aa admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:54.204548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fcc3d-3361-4b91-97d2-85006f6dff0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:54.205571 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fcc3d-3361-4b91-97d2-85006f6dff0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:54.205997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fcc3d-3361-4b91-97d2-85006f6dff0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:54.206409 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fcc3d-3361-4b91-97d2-85006f6dff0f admin admin] PATH_INFO: `/v3/projects/cded9e11aba44bbd9c7140de87e0ec63` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:54.207984 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fcc3d-3361-4b91-97d2-85006f6dff0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cded9e11aba44bbd9c7140de87e0ec63)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:54.209078 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 945/1917] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:54 2022] PUT /identity/v3/projects/8454f4c13e654ee58e6ff6265b7b85f9/users/c6591fd1ff63406c86b847870570e56c/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:54.211434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fcc3d-3361-4b91-97d2-85006f6dff0f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:54.214464 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ce8ab-733f-49c1-a16b-7f4d05a1a42e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:54.214865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ce8ab-733f-49c1-a16b-7f4d05a1a42e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:54.215147 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ce8ab-733f-49c1-a16b-7f4d05a1a42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:54.237714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e98ce8ab-733f-49c1-a16b-7f4d05a1a42e None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:54.239381 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e98ce8ab-733f-49c1-a16b-7f4d05a1a42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:54.247427 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:54.281748 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-454fcc3d-3361-4b91-97d2-85006f6dff0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cded9e11aba44bbd9c7140de87e0ec63', 'request_id': 'req-454fcc3d-3361-4b91-97d2-85006f6dff0f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:54.288990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-454fcc3d-3361-4b91-97d2-85006f6dff0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cded9e11aba44bbd9c7140de87e0ec63', 'request_id': 'req-454fcc3d-3361-4b91-97d2-85006f6dff0f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:54.296679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-454fcc3d-3361-4b91-97d2-85006f6dff0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cded9e11aba44bbd9c7140de87e0ec63', 'request_id': 'req-454fcc3d-3361-4b91-97d2-85006f6dff0f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:54.313581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-454fcc3d-3361-4b91-97d2-85006f6dff0f admin admin] The token cache is being invalidate because project cded9e11aba44bbd9c7140de87e0ec63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:54.313975 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-454fcc3d-3361-4b91-97d2-85006f6dff0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:54.314665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-454fcc3d-3361-4b91-97d2-85006f6dff0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:54.316508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 973/1918] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:54 2022] DELETE /identity/v3/projects/cded9e11aba44bbd9c7140de87e0ec63 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:34:54.321792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0de36-4722-4b56-bfa2-dfe99ae7da42 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:54.322103 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0de36-4722-4b56-bfa2-dfe99ae7da42 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:54.322405 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0de36-4722-4b56-bfa2-dfe99ae7da42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:54.325322 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 946/1919] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:54.338636 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ce8ab-733f-49c1-a16b-7f4d05a1a42e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:54.341075 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e98ce8ab-733f-49c1-a16b-7f4d05a1a42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:54.346826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-6de0de36-4722-4b56-bfa2-dfe99ae7da42 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:54.348068 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6de0de36-4722-4b56-bfa2-dfe99ae7da42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:54.401003 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:54.423027 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ce8ab-733f-49c1-a16b-7f4d05a1a42e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:54.431999 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 974/1920] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:34:54.443178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e36eb-b3b6-402b-9f7b-8084551404fd None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:54.443623 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e36eb-b3b6-402b-9f7b-8084551404fd None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:54.444136 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e36eb-b3b6-402b-9f7b-8084551404fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:54.470083 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-801e36eb-b3b6-402b-9f7b-8084551404fd None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:54.471362 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-801e36eb-b3b6-402b-9f7b-8084551404fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:54.482772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 975/1921] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:34:54.490187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e36eb-b3b6-402b-9f7b-8084551404fd None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:54.504083 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c584e88-995e-496f-9a8a-499ea2111eba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:54.510345 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e36eb-b3b6-402b-9f7b-8084551404fd None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:54.510493 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c584e88-995e-496f-9a8a-499ea2111eba service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:54.510493 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c584e88-995e-496f-9a8a-499ea2111eba service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:54.510493 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c584e88-995e-496f-9a8a-499ea2111eba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:54.511248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c203d74-65e0-41a6-86b6-66a2f18bf454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:54.512426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c203d74-65e0-41a6-86b6-66a2f18bf454 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:54.512714 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c203d74-65e0-41a6-86b6-66a2f18bf454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:54.513048 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c203d74-65e0-41a6-86b6-66a2f18bf454 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:54.516161 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c203d74-65e0-41a6-86b6-66a2f18bf454 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:34:54.516836 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c203d74-65e0-41a6-86b6-66a2f18bf454 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:54.520553 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c584e88-995e-496f-9a8a-499ea2111eba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:54.522012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c584e88-995e-496f-9a8a-499ea2111eba service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:54.522580 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c203d74-65e0-41a6-86b6-66a2f18bf454 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:54.550582 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 947/1922] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:34:54.551161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c584e88-995e-496f-9a8a-499ea2111eba service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:54.551609 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 976/1923] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:34:54.567310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c584e88-995e-496f-9a8a-499ea2111eba service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:54.573935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f9fbe-8d9d-43c1-9614-5784406d6c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:54.575007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f9fbe-8d9d-43c1-9614-5784406d6c44 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:54.575347 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f9fbe-8d9d-43c1-9614-5784406d6c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:54.575573 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f9fbe-8d9d-43c1-9614-5784406d6c44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:54.576699 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f9fbe-8d9d-43c1-9614-5784406d6c44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:54.579200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f9fbe-8d9d-43c1-9614-5784406d6c44 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:54.637138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 948/1924] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:34:54 2022] POST /identity/v3/projects => generated 361 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:54.638040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c203d74-65e0-41a6-86b6-66a2f18bf454 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:54.654873 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c203d74-65e0-41a6-86b6-66a2f18bf454 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:54.757514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8f2db-da23-4643-b04a-17f38da360b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:54.758961 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8f2db-da23-4643-b04a-17f38da360b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:54.759400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8f2db-da23-4643-b04a-17f38da360b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:54.759853 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8f2db-da23-4643-b04a-17f38da360b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:54.761481 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8f2db-da23-4643-b04a-17f38da360b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:54.763962 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8f2db-da23-4643-b04a-17f38da360b5 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:54.792286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 977/1925] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:34:54 2022] POST /identity/v3/users => generated 281 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:34:54.800007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f9fbe-8d9d-43c1-9614-5784406d6c44 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:54.820083 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f9fbe-8d9d-43c1-9614-5784406d6c44 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:54.826895 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61d53b-f414-4777-9b1b-23ecf5039d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:54.828921 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61d53b-f414-4777-9b1b-23ecf5039d3f admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:54.829470 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61d53b-f414-4777-9b1b-23ecf5039d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:54.830104 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61d53b-f414-4777-9b1b-23ecf5039d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:54.831647 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61d53b-f414-4777-9b1b-23ecf5039d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:54.833823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61d53b-f414-4777-9b1b-23ecf5039d3f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:54.864414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 949/1926] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:54.865973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8f2db-da23-4643-b04a-17f38da360b5 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:54.891999 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8f2db-da23-4643-b04a-17f38da360b5 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:54.914746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af6ab9-9423-4807-b62d-8ac4e4afb7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:54.915407 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af6ab9-9423-4807-b62d-8ac4e4afb7ff admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:54.915731 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af6ab9-9423-4807-b62d-8ac4e4afb7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:54.916004 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af6ab9-9423-4807-b62d-8ac4e4afb7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:54.916719 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af6ab9-9423-4807-b62d-8ac4e4afb7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:54.918390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af6ab9-9423-4807-b62d-8ac4e4afb7ff admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:54.952728 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 978/1927] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:54.956108 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61d53b-f414-4777-9b1b-23ecf5039d3f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:54.980499 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61d53b-f414-4777-9b1b-23ecf5039d3f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:54.998917 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f677b-7e4f-4b54-bde6-e9bd3d4e050f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:54.999624 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f677b-7e4f-4b54-bde6-e9bd3d4e050f admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:54.999913 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f677b-7e4f-4b54-bde6-e9bd3d4e050f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:55.000188 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f677b-7e4f-4b54-bde6-e9bd3d4e050f admin admin] PATH_INFO: `/v3/projects/60ecea1834a24a2388668ad671fe1b27/users/40bb91e61625420997485d7409214bda/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:55.049070 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f677b-7e4f-4b54-bde6-e9bd3d4e050f admin admin] RBAC: Authorizing `identity:create_grant(project_id=60ecea1834a24a2388668ad671fe1b27, user_id=40bb91e61625420997485d7409214bda, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:55.054300 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f677b-7e4f-4b54-bde6-e9bd3d4e050f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:55.105521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 950/1928] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:54 2022] PUT /identity/v3/projects/60ecea1834a24a2388668ad671fe1b27/users/40bb91e61625420997485d7409214bda/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:55.117666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2abcd8-03c0-4d74-b26a-0983cc748dbc None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:55.117666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2abcd8-03c0-4d74-b26a-0983cc748dbc None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:55.117666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2abcd8-03c0-4d74-b26a-0983cc748dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:55.165954 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-ba2abcd8-03c0-4d74-b26a-0983cc748dbc None None] MFA Rules not processed for user `40bb91e61625420997485d7409214bda`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:55.167833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ba2abcd8-03c0-4d74-b26a-0983cc748dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:55.290736 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 979/1929] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:55.457805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f677b-7e4f-4b54-bde6-e9bd3d4e050f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:55.460727 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e86f677b-7e4f-4b54-bde6-e9bd3d4e050f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:55.478538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:55.552035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f677b-7e4f-4b54-bde6-e9bd3d4e050f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:55.556150 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda65ad-5ee2-429b-afe0-f8ca8522475a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:55.556846 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda65ad-5ee2-429b-afe0-f8ca8522475a service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:55.557144 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda65ad-5ee2-429b-afe0-f8ca8522475a service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:55.557428 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda65ad-5ee2-429b-afe0-f8ca8522475a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:55.569506 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda65ad-5ee2-429b-afe0-f8ca8522475a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:55.571034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda65ad-5ee2-429b-afe0-f8ca8522475a service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:55.582045 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 951/1930] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:55.600699 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2abcd8-03c0-4d74-b26a-0983cc748dbc None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:55.650489 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:55.698463 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9bf6c-505d-42dd-a7da-24d6e88bc364 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:55.699213 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9bf6c-505d-42dd-a7da-24d6e88bc364 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:55.699748 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9bf6c-505d-42dd-a7da-24d6e88bc364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:55.705958 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2abcd8-03c0-4d74-b26a-0983cc748dbc None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:55.713627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c5c2a-8c6a-4794-afb5-fa12eddfe0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:55.714350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c5c2a-8c6a-4794-afb5-fa12eddfe0b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:55.714636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c5c2a-8c6a-4794-afb5-fa12eddfe0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:55.714913 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c5c2a-8c6a-4794-afb5-fa12eddfe0b4 admin admin] PATH_INFO: `/v3/users/40bb91e61625420997485d7409214bda` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:55.716208 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c5c2a-8c6a-4794-afb5-fa12eddfe0b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40bb91e61625420997485d7409214bda)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:55.718796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c5c2a-8c6a-4794-afb5-fa12eddfe0b4 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:55.738422 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-17d9bf6c-505d-42dd-a7da-24d6e88bc364 None None] MFA Rules not processed for user `c6591fd1ff63406c86b847870570e56c`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:55.740117 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-17d9bf6c-505d-42dd-a7da-24d6e88bc364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:55.796383 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-3f3c5c2a-8c6a-4794-afb5-fa12eddfe0b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40bb91e61625420997485d7409214bda'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:55.797702 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-3f3c5c2a-8c6a-4794-afb5-fa12eddfe0b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40bb91e61625420997485d7409214bda'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:55.811891 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-3f3c5c2a-8c6a-4794-afb5-fa12eddfe0b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40bb91e61625420997485d7409214bda'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:55.842862 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-3f3c5c2a-8c6a-4794-afb5-fa12eddfe0b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40bb91e61625420997485d7409214bda'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:55.862444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 952/1931] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:34:55.868195 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-3f3c5c2a-8c6a-4794-afb5-fa12eddfe0b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40bb91e61625420997485d7409214bda'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:55.874268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9bf6c-505d-42dd-a7da-24d6e88bc364 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:55.878266 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 980/1932] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:34:55 2022] DELETE /identity/v3/users/40bb91e61625420997485d7409214bda => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:55.881789 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a850d49-f4ee-4203-93be-c19dd2b2bb03 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:55.882180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a850d49-f4ee-4203-93be-c19dd2b2bb03 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:55.883019 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a850d49-f4ee-4203-93be-c19dd2b2bb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:55.897966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9bf6c-505d-42dd-a7da-24d6e88bc364 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:55.934954 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-7a850d49-f4ee-4203-93be-c19dd2b2bb03 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:55.937087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7a850d49-f4ee-4203-93be-c19dd2b2bb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:56.056003 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c450c3-820e-49fa-b397-8b45e16aadbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:56.056906 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c450c3-820e-49fa-b397-8b45e16aadbb service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:56.057193 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c450c3-820e-49fa-b397-8b45e16aadbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:56.057464 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c450c3-820e-49fa-b397-8b45e16aadbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:56.061927 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-33c450c3-820e-49fa-b397-8b45e16aadbb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:56.093995 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 981/1933] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:56.101755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a850d49-f4ee-4203-93be-c19dd2b2bb03 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:56.103520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7a850d49-f4ee-4203-93be-c19dd2b2bb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:56.155375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c450c3-820e-49fa-b397-8b45e16aadbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:56.157363 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c450c3-820e-49fa-b397-8b45e16aadbb service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:56.162188 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:56.170650 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 953/1934] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:56.175428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a850d49-f4ee-4203-93be-c19dd2b2bb03 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:56.180004 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3730c61-2245-4b17-a76d-13b9511f1e6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:56.180204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3730c61-2245-4b17-a76d-13b9511f1e6d service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:56.180454 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3730c61-2245-4b17-a76d-13b9511f1e6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:56.180731 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3730c61-2245-4b17-a76d-13b9511f1e6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:56.189821 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:56.192895 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3730c61-2245-4b17-a76d-13b9511f1e6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:56.195110 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3730c61-2245-4b17-a76d-13b9511f1e6d service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:56.217951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 982/1935] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:34:56.631911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c450c3-820e-49fa-b397-8b45e16aadbb service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:56.633468 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-33c450c3-820e-49fa-b397-8b45e16aadbb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:56.676086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c450c3-820e-49fa-b397-8b45e16aadbb service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:56.690277 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11fef9a-b30c-4172-ae0a-13e06707b70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:56.691694 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fef9a-b30c-4172-ae0a-13e06707b70b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:56.691989 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fef9a-b30c-4172-ae0a-13e06707b70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:56.692265 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fef9a-b30c-4172-ae0a-13e06707b70b admin admin] PATH_INFO: `/v3/projects/60ecea1834a24a2388668ad671fe1b27` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:56.694378 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11fef9a-b30c-4172-ae0a-13e06707b70b admin admin] RBAC: Authorizing `identity:delete_project(project_id=60ecea1834a24a2388668ad671fe1b27)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:56.699084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11fef9a-b30c-4172-ae0a-13e06707b70b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:56.804974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d11fef9a-b30c-4172-ae0a-13e06707b70b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60ecea1834a24a2388668ad671fe1b27', 'request_id': 'req-d11fef9a-b30c-4172-ae0a-13e06707b70b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:56.813524 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d11fef9a-b30c-4172-ae0a-13e06707b70b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60ecea1834a24a2388668ad671fe1b27', 'request_id': 'req-d11fef9a-b30c-4172-ae0a-13e06707b70b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:56.820366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d11fef9a-b30c-4172-ae0a-13e06707b70b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60ecea1834a24a2388668ad671fe1b27', 'request_id': 'req-d11fef9a-b30c-4172-ae0a-13e06707b70b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:56.835440 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d11fef9a-b30c-4172-ae0a-13e06707b70b admin admin] The token cache is being invalidate because project 60ecea1834a24a2388668ad671fe1b27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:56.835440 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d11fef9a-b30c-4172-ae0a-13e06707b70b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:56.836142 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d11fef9a-b30c-4172-ae0a-13e06707b70b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:56.837916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 954/1936] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:56 2022] DELETE /identity/v3/projects/60ecea1834a24a2388668ad671fe1b27 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:34:56.844912 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f44a3-8d0d-48da-a0fb-e1413acb6a8a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:56.845532 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f44a3-8d0d-48da-a0fb-e1413acb6a8a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:56.846062 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f44a3-8d0d-48da-a0fb-e1413acb6a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:56.867691 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a90f44a3-8d0d-48da-a0fb-e1413acb6a8a None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:56.869020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a90f44a3-8d0d-48da-a0fb-e1413acb6a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:56.958796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 983/1937] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:56.967667 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86012e3-19fd-44b8-b829-0044a58bbe5d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:56.968101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86012e3-19fd-44b8-b829-0044a58bbe5d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:56.968567 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86012e3-19fd-44b8-b829-0044a58bbe5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:56.992550 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a86012e3-19fd-44b8-b829-0044a58bbe5d None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:56.993955 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a86012e3-19fd-44b8-b829-0044a58bbe5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:57.005731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 955/1938] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:57.010837 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f44a3-8d0d-48da-a0fb-e1413acb6a8a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:57.023717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f44a3-8d0d-48da-a0fb-e1413acb6a8a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:57.031452 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ad7a3-0f13-4fd1-9e46-aa0d0b3125a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:57.032004 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ad7a3-0f13-4fd1-9e46-aa0d0b3125a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:57.032287 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ad7a3-0f13-4fd1-9e46-aa0d0b3125a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:57.033027 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ad7a3-0f13-4fd1-9e46-aa0d0b3125a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:57.033259 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ad7a3-0f13-4fd1-9e46-aa0d0b3125a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:34:57.033568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ad7a3-0f13-4fd1-9e46-aa0d0b3125a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:57.037669 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ad7a3-0f13-4fd1-9e46-aa0d0b3125a2 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:57.057936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 984/1939] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:34:57.065250 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86012e3-19fd-44b8-b829-0044a58bbe5d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:57.077002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86012e3-19fd-44b8-b829-0044a58bbe5d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:57.082850 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43245ad-7fd1-4c85-9164-8c18a0cca534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:57.083515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43245ad-7fd1-4c85-9164-8c18a0cca534 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:57.084020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43245ad-7fd1-4c85-9164-8c18a0cca534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:57.084311 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43245ad-7fd1-4c85-9164-8c18a0cca534 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:57.085263 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43245ad-7fd1-4c85-9164-8c18a0cca534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:57.092659 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43245ad-7fd1-4c85-9164-8c18a0cca534 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:57.123816 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 956/1940] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:34:57 2022] POST /identity/v3/projects => generated 369 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:34:57.128523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ad7a3-0f13-4fd1-9e46-aa0d0b3125a2 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:57.141064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ad7a3-0f13-4fd1-9e46-aa0d0b3125a2 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:57.234773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7685e4-8d57-495e-9474-c9179c0c85bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:57.236934 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7685e4-8d57-495e-9474-c9179c0c85bc admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:57.236934 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7685e4-8d57-495e-9474-c9179c0c85bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:57.237213 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7685e4-8d57-495e-9474-c9179c0c85bc admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:57.238011 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7685e4-8d57-495e-9474-c9179c0c85bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:57.239999 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7685e4-8d57-495e-9474-c9179c0c85bc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:57.269859 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 985/1941] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:34:57 2022] POST /identity/v3/users => generated 285 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:34:57.274069 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43245ad-7fd1-4c85-9164-8c18a0cca534 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:57.286407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43245ad-7fd1-4c85-9164-8c18a0cca534 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:57.293318 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782879b-ae9a-4d90-8b87-1d76144fc30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:57.293980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782879b-ae9a-4d90-8b87-1d76144fc30d admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:57.294905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782879b-ae9a-4d90-8b87-1d76144fc30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:57.295093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782879b-ae9a-4d90-8b87-1d76144fc30d admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:57.295803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b782879b-ae9a-4d90-8b87-1d76144fc30d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:57.297750 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b782879b-ae9a-4d90-8b87-1d76144fc30d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:57.313955 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 957/1942] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:57.315452 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7685e4-8d57-495e-9474-c9179c0c85bc admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:57.327796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7685e4-8d57-495e-9474-c9179c0c85bc admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:57.336629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a841a-de9f-4fbd-8a8b-f7fe4eedcff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:57.337424 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a841a-de9f-4fbd-8a8b-f7fe4eedcff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:57.337715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a841a-de9f-4fbd-8a8b-f7fe4eedcff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:57.338118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a841a-de9f-4fbd-8a8b-f7fe4eedcff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:57.338893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a841a-de9f-4fbd-8a8b-f7fe4eedcff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:57.341621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a841a-de9f-4fbd-8a8b-f7fe4eedcff8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:57.353418 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 986/1943] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:34:57.358392 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782879b-ae9a-4d90-8b87-1d76144fc30d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:57.368781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782879b-ae9a-4d90-8b87-1d76144fc30d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:57.374833 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760066c-f210-4f5a-b390-6cd3e4dc4a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:57.375610 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760066c-f210-4f5a-b390-6cd3e4dc4a60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:57.375969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760066c-f210-4f5a-b390-6cd3e4dc4a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:57.376310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760066c-f210-4f5a-b390-6cd3e4dc4a60 admin admin] PATH_INFO: `/v3/projects/8396d8e73bc64e688edf7cb10bea233b/users/aee1d58902994c5683b1936fc6df756c/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:57.396014 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760066c-f210-4f5a-b390-6cd3e4dc4a60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8396d8e73bc64e688edf7cb10bea233b, user_id=aee1d58902994c5683b1936fc6df756c, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:57.400584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760066c-f210-4f5a-b390-6cd3e4dc4a60 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:57.418830 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 958/1944] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:34:57 2022] PUT /identity/v3/projects/8396d8e73bc64e688edf7cb10bea233b/users/aee1d58902994c5683b1936fc6df756c/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:57.435401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da21d2-0eff-4b43-af74-dcbfe4b8152f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:57.436497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da21d2-0eff-4b43-af74-dcbfe4b8152f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:57.436961 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da21d2-0eff-4b43-af74-dcbfe4b8152f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:57.472731 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-16da21d2-0eff-4b43-af74-dcbfe4b8152f None None] MFA Rules not processed for user `aee1d58902994c5683b1936fc6df756c`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:57.473872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-16da21d2-0eff-4b43-af74-dcbfe4b8152f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:57.540702 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:57.590480 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 987/1945] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:57 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:57.600800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760066c-f210-4f5a-b390-6cd3e4dc4a60 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:57.604767 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4760066c-f210-4f5a-b390-6cd3e4dc4a60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:57.731827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760066c-f210-4f5a-b390-6cd3e4dc4a60 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:57.736058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a2d31-e014-4976-a9ad-ec8428d8a881 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:57.736358 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a2d31-e014-4976-a9ad-ec8428d8a881 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:57.736625 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a2d31-e014-4976-a9ad-ec8428d8a881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:57.738486 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdfdf3-a73a-4739-bf17-e68fcd04ad05 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69d216a568b546589e854c2b4476efe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:57.739544 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdfdf3-a73a-4739-bf17-e68fcd04ad05 service glance] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:57.740013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdfdf3-a73a-4739-bf17-e68fcd04ad05 service glance] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:57.740746 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdfdf3-a73a-4739-bf17-e68fcd04ad05 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:57.757336 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdfdf3-a73a-4739-bf17-e68fcd04ad05 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:57.757689 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c73a2d31-e014-4976-a9ad-ec8428d8a881 None None] MFA Rules not processed for user `c6591fd1ff63406c86b847870570e56c`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:57.758977 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c73a2d31-e014-4976-a9ad-ec8428d8a881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:57.759846 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdfdf3-a73a-4739-bf17-e68fcd04ad05 service glance] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:57.771251 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:57.783544 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 959/1946] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:57 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:57.854722 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 988/1947] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:34:58.072464 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdfdf3-a73a-4739-bf17-e68fcd04ad05 service glance] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:58.074007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b3cdfdf3-a73a-4739-bf17-e68fcd04ad05 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:58.160736 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdfdf3-a73a-4739-bf17-e68fcd04ad05 service glance] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:58.167014 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cb317-1251-464c-9eeb-ad1c9d859401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:58.167357 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cb317-1251-464c-9eeb-ad1c9d859401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:58.167647 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cb317-1251-464c-9eeb-ad1c9d859401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:58.167928 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cb317-1251-464c-9eeb-ad1c9d859401 admin admin] PATH_INFO: `/v3/users/319bbc3f15a64d808fcb4b7c35d393b2` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:58.170360 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400cb317-1251-464c-9eeb-ad1c9d859401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=319bbc3f15a64d808fcb4b7c35d393b2)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:58.173104 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400cb317-1251-464c-9eeb-ad1c9d859401 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:58.253590 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-400cb317-1251-464c-9eeb-ad1c9d859401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '319bbc3f15a64d808fcb4b7c35d393b2'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:58.254948 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-400cb317-1251-464c-9eeb-ad1c9d859401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '319bbc3f15a64d808fcb4b7c35d393b2'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:58.265447 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-400cb317-1251-464c-9eeb-ad1c9d859401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '319bbc3f15a64d808fcb4b7c35d393b2'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:58.292184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-400cb317-1251-464c-9eeb-ad1c9d859401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '319bbc3f15a64d808fcb4b7c35d393b2'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:58.293777 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-400cb317-1251-464c-9eeb-ad1c9d859401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '319bbc3f15a64d808fcb4b7c35d393b2'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:58.315677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 960/1948] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:34:58 2022] DELETE /identity/v3/users/319bbc3f15a64d808fcb4b7c35d393b2 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:58.317340 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50572927-6453-4c23-aec4-f6c68df2492f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:58.317864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50572927-6453-4c23-aec4-f6c68df2492f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:58.318009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50572927-6453-4c23-aec4-f6c68df2492f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:58.366973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-50572927-6453-4c23-aec4-f6c68df2492f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:58.368259 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-50572927-6453-4c23-aec4-f6c68df2492f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:58.473963 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 989/1949] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:34:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:58.486767 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cb317-1251-464c-9eeb-ad1c9d859401 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:58.488562 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-400cb317-1251-464c-9eeb-ad1c9d859401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:58.554654 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cb317-1251-464c-9eeb-ad1c9d859401 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:58.558474 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8afa28-b6f5-42b8-b65b-e757beceb2d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:58.559243 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8afa28-b6f5-42b8-b65b-e757beceb2d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:58.559598 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8afa28-b6f5-42b8-b65b-e757beceb2d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:58.559966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8afa28-b6f5-42b8-b65b-e757beceb2d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:58.570345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8afa28-b6f5-42b8-b65b-e757beceb2d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:58.571924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8afa28-b6f5-42b8-b65b-e757beceb2d0 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:58.582414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 961/1950] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:58.905513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50572927-6453-4c23-aec4-f6c68df2492f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:58.907679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-50572927-6453-4c23-aec4-f6c68df2492f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:58.941035 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50572927-6453-4c23-aec4-f6c68df2492f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:58.946280 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8db196-a661-4624-b94c-8a782c779d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:58.947074 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8db196-a661-4624-b94c-8a782c779d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:58.947358 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8db196-a661-4624-b94c-8a782c779d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:58.947633 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8db196-a661-4624-b94c-8a782c779d82 admin admin] PATH_INFO: `/v3/projects/b21f0dea9004491eaae324a40b339670` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:58.948822 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8db196-a661-4624-b94c-8a782c779d82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b21f0dea9004491eaae324a40b339670)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:58.951272 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8db196-a661-4624-b94c-8a782c779d82 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:59.009868 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2c8db196-a661-4624-b94c-8a782c779d82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b21f0dea9004491eaae324a40b339670', 'request_id': 'req-2c8db196-a661-4624-b94c-8a782c779d82'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:59.016687 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2c8db196-a661-4624-b94c-8a782c779d82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b21f0dea9004491eaae324a40b339670', 'request_id': 'req-2c8db196-a661-4624-b94c-8a782c779d82'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:59.024112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2c8db196-a661-4624-b94c-8a782c779d82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b21f0dea9004491eaae324a40b339670', 'request_id': 'req-2c8db196-a661-4624-b94c-8a782c779d82'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:59.047485 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2c8db196-a661-4624-b94c-8a782c779d82 admin admin] The token cache is being invalidate because project b21f0dea9004491eaae324a40b339670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:59.048057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2c8db196-a661-4624-b94c-8a782c779d82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:59.048956 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2c8db196-a661-4624-b94c-8a782c779d82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:59.051324 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 990/1951] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:58 2022] DELETE /identity/v3/projects/b21f0dea9004491eaae324a40b339670 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:59.055230 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8afa28-b6f5-42b8-b65b-e757beceb2d0 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:59.068137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8afa28-b6f5-42b8-b65b-e757beceb2d0 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:59.091958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:59.145452 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c899dc1-9385-4f1a-ad8d-a99427986f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:59.146349 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c899dc1-9385-4f1a-ad8d-a99427986f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:59.146637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c899dc1-9385-4f1a-ad8d-a99427986f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:59.146908 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c899dc1-9385-4f1a-ad8d-a99427986f1c admin admin] PATH_INFO: `/v3/users/7cf52d5f3f164a61b7b57ac4415408d4` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:59.148124 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c899dc1-9385-4f1a-ad8d-a99427986f1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cf52d5f3f164a61b7b57ac4415408d4)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:59.150130 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c899dc1-9385-4f1a-ad8d-a99427986f1c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:59.208476 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3c899dc1-9385-4f1a-ad8d-a99427986f1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cf52d5f3f164a61b7b57ac4415408d4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:59.209215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3c899dc1-9385-4f1a-ad8d-a99427986f1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cf52d5f3f164a61b7b57ac4415408d4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:59.220223 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3c899dc1-9385-4f1a-ad8d-a99427986f1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cf52d5f3f164a61b7b57ac4415408d4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:59.253875 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3c899dc1-9385-4f1a-ad8d-a99427986f1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cf52d5f3f164a61b7b57ac4415408d4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:59.264350 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3c899dc1-9385-4f1a-ad8d-a99427986f1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cf52d5f3f164a61b7b57ac4415408d4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:59.296949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 962/1952] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:34:59 2022] DELETE /identity/v3/users/7cf52d5f3f164a61b7b57ac4415408d4 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:59.844204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8db196-a661-4624-b94c-8a782c779d82 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:59.858219 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2c8db196-a661-4624-b94c-8a782c779d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:59.884650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8db196-a661-4624-b94c-8a782c779d82 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:59.910718 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:00.044630 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbadc1-2986-4029-a468-69a29778a3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:00.046358 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbadc1-2986-4029-a468-69a29778a3ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:00.046504 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbadc1-2986-4029-a468-69a29778a3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:00.046937 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbadc1-2986-4029-a468-69a29778a3ab admin admin] PATH_INFO: `/v3/projects/740a8982fd7f48768193372fa069ea35` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:00.049068 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bbadc1-2986-4029-a468-69a29778a3ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=740a8982fd7f48768193372fa069ea35)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:00.052877 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bbadc1-2986-4029-a468-69a29778a3ab admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:00.126270 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f0bbadc1-2986-4029-a468-69a29778a3ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '740a8982fd7f48768193372fa069ea35', 'request_id': 'req-f0bbadc1-2986-4029-a468-69a29778a3ab'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:00.148895 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f0bbadc1-2986-4029-a468-69a29778a3ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '740a8982fd7f48768193372fa069ea35', 'request_id': 'req-f0bbadc1-2986-4029-a468-69a29778a3ab'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:00.159635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f0bbadc1-2986-4029-a468-69a29778a3ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '740a8982fd7f48768193372fa069ea35', 'request_id': 'req-f0bbadc1-2986-4029-a468-69a29778a3ab'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:00.181331 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f0bbadc1-2986-4029-a468-69a29778a3ab admin admin] The token cache is being invalidate because project 740a8982fd7f48768193372fa069ea35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:35:00.181718 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f0bbadc1-2986-4029-a468-69a29778a3ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:00.182497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f0bbadc1-2986-4029-a468-69a29778a3ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:00.185940 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 991/1953] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:34:59 2022] DELETE /identity/v3/projects/740a8982fd7f48768193372fa069ea35 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:00.189689 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb62a3-5728-4747-b680-3e873ea68772 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:00.190229 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb62a3-5728-4747-b680-3e873ea68772 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:00.190436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb62a3-5728-4747-b680-3e873ea68772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:00.214945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c9eb62a3-5728-4747-b680-3e873ea68772 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:00.215334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c9eb62a3-5728-4747-b680-3e873ea68772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:00.297952 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 963/1954] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:00.309678 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a5c5c-c10e-43a3-a805-822ad11eef96 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:00.310846 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a5c5c-c10e-43a3-a805-822ad11eef96 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:00.311318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a5c5c-c10e-43a3-a805-822ad11eef96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:00.336438 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-ed4a5c5c-c10e-43a3-a805-822ad11eef96 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:00.337679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ed4a5c5c-c10e-43a3-a805-822ad11eef96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:00.346802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 992/1955] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:00.353378 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb62a3-5728-4747-b680-3e873ea68772 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:00.366272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb62a3-5728-4747-b680-3e873ea68772 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:00.369770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f57d2a-2868-4405-b70d-810b5a81be67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:00.370536 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f57d2a-2868-4405-b70d-810b5a81be67 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:00.370903 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f57d2a-2868-4405-b70d-810b5a81be67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:00.371963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f57d2a-2868-4405-b70d-810b5a81be67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:00.373547 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f57d2a-2868-4405-b70d-810b5a81be67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:35:00.373942 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f57d2a-2868-4405-b70d-810b5a81be67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:00.376525 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f57d2a-2868-4405-b70d-810b5a81be67 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:00.396682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 964/1956] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:35:00.401843 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a5c5c-c10e-43a3-a805-822ad11eef96 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:00.414345 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a5c5c-c10e-43a3-a805-822ad11eef96 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:00.422859 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8731dff-de5e-4566-af6d-1f80df139613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:00.425283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8731dff-de5e-4566-af6d-1f80df139613 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:00.426298 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8731dff-de5e-4566-af6d-1f80df139613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:00.426813 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8731dff-de5e-4566-af6d-1f80df139613 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:00.427833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8731dff-de5e-4566-af6d-1f80df139613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:00.430203 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8731dff-de5e-4566-af6d-1f80df139613 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:00.472547 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 993/1957] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:35:00 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:00.478169 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f57d2a-2868-4405-b70d-810b5a81be67 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:00.492143 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f57d2a-2868-4405-b70d-810b5a81be67 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:00.565928 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:00.625442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbcb58-7235-493d-aee2-cbe2521a3aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:00.627925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbcb58-7235-493d-aee2-cbe2521a3aff admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:00.628264 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbcb58-7235-493d-aee2-cbe2521a3aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:00.628544 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbcb58-7235-493d-aee2-cbe2521a3aff admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:00.630874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbcb58-7235-493d-aee2-cbe2521a3aff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:00.634082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbcb58-7235-493d-aee2-cbe2521a3aff admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:00.667979 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 965/1958] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:35:00 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:00.672640 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8731dff-de5e-4566-af6d-1f80df139613 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:00.685349 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8731dff-de5e-4566-af6d-1f80df139613 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:00.692294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30db4cf-6748-4524-b4ce-55eec89423a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:00.693851 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30db4cf-6748-4524-b4ce-55eec89423a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:00.694508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30db4cf-6748-4524-b4ce-55eec89423a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:00.694813 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30db4cf-6748-4524-b4ce-55eec89423a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:00.695727 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30db4cf-6748-4524-b4ce-55eec89423a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:00.697500 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30db4cf-6748-4524-b4ce-55eec89423a1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:00.710824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 994/1959] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:35:00.714941 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbcb58-7235-493d-aee2-cbe2521a3aff admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:00.726214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbcb58-7235-493d-aee2-cbe2521a3aff admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:00.731491 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30338b02-40a3-4f76-9780-2447df9e1b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:00.733021 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30338b02-40a3-4f76-9780-2447df9e1b61 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:00.733412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30338b02-40a3-4f76-9780-2447df9e1b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:00.733764 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30338b02-40a3-4f76-9780-2447df9e1b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:00.735077 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30338b02-40a3-4f76-9780-2447df9e1b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:00.736998 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30338b02-40a3-4f76-9780-2447df9e1b61 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:00.750921 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 966/1960] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:00.755970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30db4cf-6748-4524-b4ce-55eec89423a1 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:00.766777 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30db4cf-6748-4524-b4ce-55eec89423a1 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:00.771815 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450bf4e-1396-4743-a7a9-8bffc02473e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:00.772573 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450bf4e-1396-4743-a7a9-8bffc02473e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:00.772929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450bf4e-1396-4743-a7a9-8bffc02473e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:00.773312 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450bf4e-1396-4743-a7a9-8bffc02473e5 admin admin] PATH_INFO: `/v3/projects/030b431cdb234b64b1aaf2b9897ff34f/users/991d1f6c0c744abc80102433dce4e5ee/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:00.790297 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450bf4e-1396-4743-a7a9-8bffc02473e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=030b431cdb234b64b1aaf2b9897ff34f, user_id=991d1f6c0c744abc80102433dce4e5ee, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:00.792992 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450bf4e-1396-4743-a7a9-8bffc02473e5 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:00.811155 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 995/1961] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:35:00 2022] PUT /identity/v3/projects/030b431cdb234b64b1aaf2b9897ff34f/users/991d1f6c0c744abc80102433dce4e5ee/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:00.828326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d3b99-0a55-43f8-a16e-0b3802d2382c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:00.828838 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d3b99-0a55-43f8-a16e-0b3802d2382c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:00.829336 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d3b99-0a55-43f8-a16e-0b3802d2382c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:00.864524 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-b51d3b99-0a55-43f8-a16e-0b3802d2382c None None] MFA Rules not processed for user `991d1f6c0c744abc80102433dce4e5ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:00.865850 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b51d3b99-0a55-43f8-a16e-0b3802d2382c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:00.956294 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 967/1962] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:00.961837 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450bf4e-1396-4743-a7a9-8bffc02473e5 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:01.027660 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:01.049143 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450bf4e-1396-4743-a7a9-8bffc02473e5 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:01.075353 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0870f-b5c4-4714-8631-fd26f05df908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:01.076468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0870f-b5c4-4714-8631-fd26f05df908 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:01.076771 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0870f-b5c4-4714-8631-fd26f05df908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:01.077049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0870f-b5c4-4714-8631-fd26f05df908 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:01.077920 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0870f-b5c4-4714-8631-fd26f05df908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:01.080609 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0870f-b5c4-4714-8631-fd26f05df908 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:01.136897 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 996/1963] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:35:00 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:35:01.141186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d3b99-0a55-43f8-a16e-0b3802d2382c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:01.157235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d3b99-0a55-43f8-a16e-0b3802d2382c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:01.232504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68037396-a355-42a7-9e9a-5032e9388d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:01.234426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68037396-a355-42a7-9e9a-5032e9388d3e admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:01.234736 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68037396-a355-42a7-9e9a-5032e9388d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:01.235015 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68037396-a355-42a7-9e9a-5032e9388d3e admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:01.235692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68037396-a355-42a7-9e9a-5032e9388d3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:01.237947 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68037396-a355-42a7-9e9a-5032e9388d3e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:01.263831 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 968/1964] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:35:01 2022] POST /identity/v3/users => generated 290 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:01.268054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0870f-b5c4-4714-8631-fd26f05df908 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:01.281218 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0870f-b5c4-4714-8631-fd26f05df908 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:01.286227 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84112a-aa8d-438e-9e54-e1860d2f1d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:01.286858 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84112a-aa8d-438e-9e54-e1860d2f1d10 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:01.287138 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84112a-aa8d-438e-9e54-e1860d2f1d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:01.287413 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84112a-aa8d-438e-9e54-e1860d2f1d10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:01.288113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84112a-aa8d-438e-9e54-e1860d2f1d10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:01.289996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84112a-aa8d-438e-9e54-e1860d2f1d10 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:01.303012 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 997/1965] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:01.307635 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68037396-a355-42a7-9e9a-5032e9388d3e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:01.319908 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68037396-a355-42a7-9e9a-5032e9388d3e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:01.325076 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed375e01-4ceb-43e3-a451-6325a91a8b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:01.325862 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed375e01-4ceb-43e3-a451-6325a91a8b1e admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:01.326187 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed375e01-4ceb-43e3-a451-6325a91a8b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:01.326522 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed375e01-4ceb-43e3-a451-6325a91a8b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:01.327282 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed375e01-4ceb-43e3-a451-6325a91a8b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:01.329325 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed375e01-4ceb-43e3-a451-6325a91a8b1e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:01.340567 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 969/1966] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:01.345428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84112a-aa8d-438e-9e54-e1860d2f1d10 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:01.355318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84112a-aa8d-438e-9e54-e1860d2f1d10 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:01.360608 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b905afc-d40b-41f8-bab6-e97bbac65a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:01.361287 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b905afc-d40b-41f8-bab6-e97bbac65a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:01.361566 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b905afc-d40b-41f8-bab6-e97bbac65a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:01.361843 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b905afc-d40b-41f8-bab6-e97bbac65a2b admin admin] PATH_INFO: `/v3/projects/25a69024165c449aa2afbf2c5c33811d/users/c720e2473c75498cada4d832fb2661b3/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:01.379806 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b905afc-d40b-41f8-bab6-e97bbac65a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a69024165c449aa2afbf2c5c33811d, user_id=c720e2473c75498cada4d832fb2661b3, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:01.382549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b905afc-d40b-41f8-bab6-e97bbac65a2b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:01.397249 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 998/1967] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:35:01 2022] PUT /identity/v3/projects/25a69024165c449aa2afbf2c5c33811d/users/c720e2473c75498cada4d832fb2661b3/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:01.408341 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ce2dc-c161-4b0c-a261-b1744446c934 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:01.408648 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ce2dc-c161-4b0c-a261-b1744446c934 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:01.408953 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ce2dc-c161-4b0c-a261-b1744446c934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:01.441422 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-5c4ce2dc-c161-4b0c-a261-b1744446c934 None None] MFA Rules not processed for user `c720e2473c75498cada4d832fb2661b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:01.442737 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5c4ce2dc-c161-4b0c-a261-b1744446c934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:01.447564 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:01.518417 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 970/1968] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:01.522800 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b905afc-d40b-41f8-bab6-e97bbac65a2b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:01.592450 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b905afc-d40b-41f8-bab6-e97bbac65a2b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:01.598314 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a85770-a8cd-4c59-99de-dc826c621161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:01.598627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a85770-a8cd-4c59-99de-dc826c621161 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:01.598915 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a85770-a8cd-4c59-99de-dc826c621161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:01.599192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a85770-a8cd-4c59-99de-dc826c621161 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:01.600050 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a85770-a8cd-4c59-99de-dc826c621161 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:01.602024 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a85770-a8cd-4c59-99de-dc826c621161 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:01.640824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 999/1969] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:35:01 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:35:01.644790 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ce2dc-c161-4b0c-a261-b1744446c934 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:01.664940 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ce2dc-c161-4b0c-a261-b1744446c934 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:01.730300 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf948c-acfa-4997-846e-84ae063e170b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:01.730936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf948c-acfa-4997-846e-84ae063e170b admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:01.731220 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf948c-acfa-4997-846e-84ae063e170b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:01.731491 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf948c-acfa-4997-846e-84ae063e170b admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:01.732689 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf948c-acfa-4997-846e-84ae063e170b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:01.734675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf948c-acfa-4997-846e-84ae063e170b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:01.764061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a85770-a8cd-4c59-99de-dc826c621161 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:01.766603 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 971/1970] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:35:01 2022] POST /identity/v3/users => generated 295 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:35:01.778208 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a85770-a8cd-4c59-99de-dc826c621161 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:01.783176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5192dc-260b-4470-aba7-2a23b7aabff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:01.783954 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5192dc-260b-4470-aba7-2a23b7aabff9 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:01.784078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5192dc-260b-4470-aba7-2a23b7aabff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:01.784349 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5192dc-260b-4470-aba7-2a23b7aabff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:01.785040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5192dc-260b-4470-aba7-2a23b7aabff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:01.786705 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5192dc-260b-4470-aba7-2a23b7aabff9 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:01.801931 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1000/1971] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:01.803948 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf948c-acfa-4997-846e-84ae063e170b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:01.816262 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf948c-acfa-4997-846e-84ae063e170b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:01.822734 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397e9f6-0c57-4903-92ed-22292e30169b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:01.823432 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397e9f6-0c57-4903-92ed-22292e30169b admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:01.823710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397e9f6-0c57-4903-92ed-22292e30169b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:01.823989 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397e9f6-0c57-4903-92ed-22292e30169b admin admin] PATH_INFO: `/v3/projects/16779c5ad60442d4b7f332bc8710792e/users/78206c1b60864da5b8740b9207fe5f63/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:01.840701 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397e9f6-0c57-4903-92ed-22292e30169b admin admin] RBAC: Authorizing `identity:create_grant(project_id=16779c5ad60442d4b7f332bc8710792e, user_id=78206c1b60864da5b8740b9207fe5f63, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:01.843515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397e9f6-0c57-4903-92ed-22292e30169b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:01.872537 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 972/1972] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:35:01 2022] PUT /identity/v3/projects/16779c5ad60442d4b7f332bc8710792e/users/78206c1b60864da5b8740b9207fe5f63/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:01.876850 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5192dc-260b-4470-aba7-2a23b7aabff9 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:01.890631 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5192dc-260b-4470-aba7-2a23b7aabff9 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:01.957598 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f55a8-7383-472c-8ad3-27af5b84bc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:01.958265 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f55a8-7383-472c-8ad3-27af5b84bc44 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:01.958680 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f55a8-7383-472c-8ad3-27af5b84bc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:01.958862 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f55a8-7383-472c-8ad3-27af5b84bc44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:01.959508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f55a8-7383-472c-8ad3-27af5b84bc44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:01.961413 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f55a8-7383-472c-8ad3-27af5b84bc44 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:01.972957 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1001/1973] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:01.976871 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397e9f6-0c57-4903-92ed-22292e30169b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:01.988491 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397e9f6-0c57-4903-92ed-22292e30169b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:01.994303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634b0d8-1037-4165-a311-8bba1c371936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:01.995950 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634b0d8-1037-4165-a311-8bba1c371936 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:01.996487 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634b0d8-1037-4165-a311-8bba1c371936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:01.996753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634b0d8-1037-4165-a311-8bba1c371936 admin admin] PATH_INFO: `/v3/projects/16779c5ad60442d4b7f332bc8710792e/users/78206c1b60864da5b8740b9207fe5f63/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:01.999492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634b0d8-1037-4165-a311-8bba1c371936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16779c5ad60442d4b7f332bc8710792e, user_id=78206c1b60864da5b8740b9207fe5f63, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:02.002916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634b0d8-1037-4165-a311-8bba1c371936 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:02.018388 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 973/1974] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:35:01 2022] PUT /identity/v3/projects/16779c5ad60442d4b7f332bc8710792e/users/78206c1b60864da5b8740b9207fe5f63/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:35:02.029515 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3a970-e188-45c1-a928-2b4308d896e0 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:02.029812 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3a970-e188-45c1-a928-2b4308d896e0 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:02.030263 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3a970-e188-45c1-a928-2b4308d896e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:02.048206 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:02.076379 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-def3a970-e188-45c1-a928-2b4308d896e0 None None] MFA Rules not processed for user `78206c1b60864da5b8740b9207fe5f63`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:02.077593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-def3a970-e188-45c1-a928-2b4308d896e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:02.157700 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1002/1975] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:02.169514 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634b0d8-1037-4165-a311-8bba1c371936 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:02.171204 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8634b0d8-1037-4165-a311-8bba1c371936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:02.231641 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634b0d8-1037-4165-a311-8bba1c371936 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:02.235339 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52205d1a-7996-4c80-a556-acd7d9c5dc20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:02.235968 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52205d1a-7996-4c80-a556-acd7d9c5dc20 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:02.236252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52205d1a-7996-4c80-a556-acd7d9c5dc20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:02.236528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52205d1a-7996-4c80-a556-acd7d9c5dc20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:02.246511 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52205d1a-7996-4c80-a556-acd7d9c5dc20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:02.248456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52205d1a-7996-4c80-a556-acd7d9c5dc20 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:02.254827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:02.260810 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 974/1976] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:35:02.306875 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3a970-e188-45c1-a928-2b4308d896e0 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:02.308342 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-def3a970-e188-45c1-a928-2b4308d896e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:02.322077 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3a970-e188-45c1-a928-2b4308d896e0 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:02.326508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f37f7a-7e9f-44c6-bda6-73433f2d0308 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:02.327147 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f37f7a-7e9f-44c6-bda6-73433f2d0308 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:02.327430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f37f7a-7e9f-44c6-bda6-73433f2d0308 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:02.327704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f37f7a-7e9f-44c6-bda6-73433f2d0308 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:02.392261 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f37f7a-7e9f-44c6-bda6-73433f2d0308 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:02.393637 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f37f7a-7e9f-44c6-bda6-73433f2d0308 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:02.402768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1003/1977] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:35:05.421035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52205d1a-7996-4c80-a556-acd7d9c5dc20 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:05.500155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52205d1a-7996-4c80-a556-acd7d9c5dc20 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:05.504858 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3361f-c878-4122-896e-946992f1ab31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:05.505519 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3361f-c878-4122-896e-946992f1ab31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:05.505806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3361f-c878-4122-896e-946992f1ab31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:05.506116 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3361f-c878-4122-896e-946992f1ab31 admin admin] PATH_INFO: `/v3/users/991d1f6c0c744abc80102433dce4e5ee` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:05.507362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3361f-c878-4122-896e-946992f1ab31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=991d1f6c0c744abc80102433dce4e5ee)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:05.509301 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3361f-c878-4122-896e-946992f1ab31 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:05.572302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e0c3361f-c878-4122-896e-946992f1ab31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '991d1f6c0c744abc80102433dce4e5ee'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:05.573164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e0c3361f-c878-4122-896e-946992f1ab31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '991d1f6c0c744abc80102433dce4e5ee'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:05.583589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e0c3361f-c878-4122-896e-946992f1ab31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '991d1f6c0c744abc80102433dce4e5ee'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:05.601254 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e0c3361f-c878-4122-896e-946992f1ab31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '991d1f6c0c744abc80102433dce4e5ee'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:05.601547 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e0c3361f-c878-4122-896e-946992f1ab31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '991d1f6c0c744abc80102433dce4e5ee'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:05.615251 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 975/1978] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:35:05 2022] DELETE /identity/v3/users/991d1f6c0c744abc80102433dce4e5ee => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:05.620326 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1d780-4e5d-4a58-9f8f-047f7cb94486 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:05.620877 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1d780-4e5d-4a58-9f8f-047f7cb94486 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:05.621033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1d780-4e5d-4a58-9f8f-047f7cb94486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:05.645116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-acc1d780-4e5d-4a58-9f8f-047f7cb94486 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:05.647048 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-acc1d780-4e5d-4a58-9f8f-047f7cb94486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:05.729367 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1004/1979] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:05.741867 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3361f-c878-4122-896e-946992f1ab31 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:05.743445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e0c3361f-c878-4122-896e-946992f1ab31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:05.804516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3361f-c878-4122-896e-946992f1ab31 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:05.808657 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbce40e-ecff-4498-8d0a-39e6c7c87231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:05.809294 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbce40e-ecff-4498-8d0a-39e6c7c87231 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:05.809607 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbce40e-ecff-4498-8d0a-39e6c7c87231 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:05.809955 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbce40e-ecff-4498-8d0a-39e6c7c87231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:05.819679 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbce40e-ecff-4498-8d0a-39e6c7c87231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:05.821140 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbce40e-ecff-4498-8d0a-39e6c7c87231 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:05.832475 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 976/1980] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:35:06.153335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1d780-4e5d-4a58-9f8f-047f7cb94486 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:06.155395 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-acc1d780-4e5d-4a58-9f8f-047f7cb94486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:06.171787 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1d780-4e5d-4a58-9f8f-047f7cb94486 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:06.176694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6df184-71a0-4251-8024-317a42ee14a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:06.177367 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6df184-71a0-4251-8024-317a42ee14a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:06.177650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6df184-71a0-4251-8024-317a42ee14a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:06.177973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6df184-71a0-4251-8024-317a42ee14a3 admin admin] PATH_INFO: `/v3/projects/030b431cdb234b64b1aaf2b9897ff34f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:06.179217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6df184-71a0-4251-8024-317a42ee14a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=030b431cdb234b64b1aaf2b9897ff34f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:06.181407 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6df184-71a0-4251-8024-317a42ee14a3 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:06.219044 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:06.245940 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ea6df184-71a0-4251-8024-317a42ee14a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '030b431cdb234b64b1aaf2b9897ff34f', 'request_id': 'req-ea6df184-71a0-4251-8024-317a42ee14a3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:06.255166 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ea6df184-71a0-4251-8024-317a42ee14a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '030b431cdb234b64b1aaf2b9897ff34f', 'request_id': 'req-ea6df184-71a0-4251-8024-317a42ee14a3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:06.262173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ea6df184-71a0-4251-8024-317a42ee14a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '030b431cdb234b64b1aaf2b9897ff34f', 'request_id': 'req-ea6df184-71a0-4251-8024-317a42ee14a3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:06.276885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ea6df184-71a0-4251-8024-317a42ee14a3 admin admin] The token cache is being invalidate because project 030b431cdb234b64b1aaf2b9897ff34f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:35:06.277175 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ea6df184-71a0-4251-8024-317a42ee14a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:06.278569 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ea6df184-71a0-4251-8024-317a42ee14a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:06.279594 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1005/1981] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:35:06 2022] DELETE /identity/v3/projects/030b431cdb234b64b1aaf2b9897ff34f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:06.284786 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbce40e-ecff-4498-8d0a-39e6c7c87231 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:06.295389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbce40e-ecff-4498-8d0a-39e6c7c87231 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:06.317217 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:06.361377 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb7c21-33a1-4766-83d1-db1fbadb9f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:06.361976 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb7c21-33a1-4766-83d1-db1fbadb9f6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:06.362271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb7c21-33a1-4766-83d1-db1fbadb9f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:06.362550 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb7c21-33a1-4766-83d1-db1fbadb9f6a admin admin] PATH_INFO: `/v3/users/c720e2473c75498cada4d832fb2661b3` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:06.363729 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb7c21-33a1-4766-83d1-db1fbadb9f6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c720e2473c75498cada4d832fb2661b3)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:06.365673 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb7c21-33a1-4766-83d1-db1fbadb9f6a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:06.416628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d8eb7c21-33a1-4766-83d1-db1fbadb9f6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c720e2473c75498cada4d832fb2661b3'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:06.417299 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d8eb7c21-33a1-4766-83d1-db1fbadb9f6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c720e2473c75498cada4d832fb2661b3'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:06.428776 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d8eb7c21-33a1-4766-83d1-db1fbadb9f6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c720e2473c75498cada4d832fb2661b3'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:06.445720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d8eb7c21-33a1-4766-83d1-db1fbadb9f6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c720e2473c75498cada4d832fb2661b3'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:06.446770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d8eb7c21-33a1-4766-83d1-db1fbadb9f6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c720e2473c75498cada4d832fb2661b3'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:06.463604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 977/1982] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:35:06 2022] DELETE /identity/v3/users/c720e2473c75498cada4d832fb2661b3 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:06.801049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6df184-71a0-4251-8024-317a42ee14a3 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:06.802603 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ea6df184-71a0-4251-8024-317a42ee14a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:06.817276 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6df184-71a0-4251-8024-317a42ee14a3 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:06.891155 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e21b37-d7b3-43f9-95ae-a3eff69f45eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:06.892635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e21b37-d7b3-43f9-95ae-a3eff69f45eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:06.892923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e21b37-d7b3-43f9-95ae-a3eff69f45eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:06.893248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e21b37-d7b3-43f9-95ae-a3eff69f45eb admin admin] PATH_INFO: `/v3/projects/25a69024165c449aa2afbf2c5c33811d` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:06.894495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e21b37-d7b3-43f9-95ae-a3eff69f45eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=25a69024165c449aa2afbf2c5c33811d)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:06.896447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e21b37-d7b3-43f9-95ae-a3eff69f45eb admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:06.963564 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a7e21b37-d7b3-43f9-95ae-a3eff69f45eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25a69024165c449aa2afbf2c5c33811d', 'request_id': 'req-a7e21b37-d7b3-43f9-95ae-a3eff69f45eb'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:06.970349 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a7e21b37-d7b3-43f9-95ae-a3eff69f45eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25a69024165c449aa2afbf2c5c33811d', 'request_id': 'req-a7e21b37-d7b3-43f9-95ae-a3eff69f45eb'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:06.977508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a7e21b37-d7b3-43f9-95ae-a3eff69f45eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25a69024165c449aa2afbf2c5c33811d', 'request_id': 'req-a7e21b37-d7b3-43f9-95ae-a3eff69f45eb'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:06.992068 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a7e21b37-d7b3-43f9-95ae-a3eff69f45eb admin admin] The token cache is being invalidate because project 25a69024165c449aa2afbf2c5c33811d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:35:06.992391 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a7e21b37-d7b3-43f9-95ae-a3eff69f45eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:06.993027 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a7e21b37-d7b3-43f9-95ae-a3eff69f45eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:06.994612 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1006/1983] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:35:06 2022] DELETE /identity/v3/projects/25a69024165c449aa2afbf2c5c33811d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:06.998602 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb7c21-33a1-4766-83d1-db1fbadb9f6a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:07.000125 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d8eb7c21-33a1-4766-83d1-db1fbadb9f6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:07.013845 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb7c21-33a1-4766-83d1-db1fbadb9f6a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:07.077299 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae80cb-b13f-471f-a292-b5a24e8a817f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:07.078143 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae80cb-b13f-471f-a292-b5a24e8a817f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:07.078431 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae80cb-b13f-471f-a292-b5a24e8a817f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:07.078708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae80cb-b13f-471f-a292-b5a24e8a817f admin admin] PATH_INFO: `/v3/users/78206c1b60864da5b8740b9207fe5f63` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:07.079985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae80cb-b13f-471f-a292-b5a24e8a817f admin admin] RBAC: Authorizing `identity:delete_user(user_id=78206c1b60864da5b8740b9207fe5f63)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:07.082045 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae80cb-b13f-471f-a292-b5a24e8a817f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:07.134700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:07.137971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-bdae80cb-b13f-471f-a292-b5a24e8a817f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78206c1b60864da5b8740b9207fe5f63'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:07.138872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-bdae80cb-b13f-471f-a292-b5a24e8a817f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78206c1b60864da5b8740b9207fe5f63'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:07.147618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-bdae80cb-b13f-471f-a292-b5a24e8a817f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78206c1b60864da5b8740b9207fe5f63'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:07.159889 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-bdae80cb-b13f-471f-a292-b5a24e8a817f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78206c1b60864da5b8740b9207fe5f63'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:07.160556 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-bdae80cb-b13f-471f-a292-b5a24e8a817f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78206c1b60864da5b8740b9207fe5f63'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:07.174130 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 978/1984] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:35:06 2022] DELETE /identity/v3/users/78206c1b60864da5b8740b9207fe5f63 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:07.567153 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e21b37-d7b3-43f9-95ae-a3eff69f45eb admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:07.568664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a7e21b37-d7b3-43f9-95ae-a3eff69f45eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:07.585284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e21b37-d7b3-43f9-95ae-a3eff69f45eb admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:07.640741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:07.667876 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e6eab-909a-403a-b33d-e0bf7a8bf1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:07.668445 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e6eab-909a-403a-b33d-e0bf7a8bf1a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:07.668730 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e6eab-909a-403a-b33d-e0bf7a8bf1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:07.669007 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e6eab-909a-403a-b33d-e0bf7a8bf1a9 admin admin] PATH_INFO: `/v3/projects/16779c5ad60442d4b7f332bc8710792e` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:07.670803 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e6eab-909a-403a-b33d-e0bf7a8bf1a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16779c5ad60442d4b7f332bc8710792e)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:07.672747 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e6eab-909a-403a-b33d-e0bf7a8bf1a9 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:07.724294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9f8e6eab-909a-403a-b33d-e0bf7a8bf1a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16779c5ad60442d4b7f332bc8710792e', 'request_id': 'req-9f8e6eab-909a-403a-b33d-e0bf7a8bf1a9'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:07.733163 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9f8e6eab-909a-403a-b33d-e0bf7a8bf1a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16779c5ad60442d4b7f332bc8710792e', 'request_id': 'req-9f8e6eab-909a-403a-b33d-e0bf7a8bf1a9'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:07.739439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9f8e6eab-909a-403a-b33d-e0bf7a8bf1a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16779c5ad60442d4b7f332bc8710792e', 'request_id': 'req-9f8e6eab-909a-403a-b33d-e0bf7a8bf1a9'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:07.754224 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9f8e6eab-909a-403a-b33d-e0bf7a8bf1a9 admin admin] The token cache is being invalidate because project 16779c5ad60442d4b7f332bc8710792e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:35:07.754518 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9f8e6eab-909a-403a-b33d-e0bf7a8bf1a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:07.755191 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9f8e6eab-909a-403a-b33d-e0bf7a8bf1a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:07.756741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1007/1985] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:35:07 2022] DELETE /identity/v3/projects/16779c5ad60442d4b7f332bc8710792e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:07.763609 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7aa35-348c-4456-81e7-943fe71e09b6 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:07.764101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7aa35-348c-4456-81e7-943fe71e09b6 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:07.764188 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7aa35-348c-4456-81e7-943fe71e09b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:07.784557 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e5e7aa35-348c-4456-81e7-943fe71e09b6 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:07.785798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e5e7aa35-348c-4456-81e7-943fe71e09b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:07.861100 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 979/1986] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:07.869582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05b924-e936-42da-b029-3c1aae634140 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:07.870180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05b924-e936-42da-b029-3c1aae634140 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:07.870386 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05b924-e936-42da-b029-3c1aae634140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:07.897147 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-6a05b924-e936-42da-b029-3c1aae634140 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:07.898718 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6a05b924-e936-42da-b029-3c1aae634140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:07.911290 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1008/1987] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:07.913176 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7aa35-348c-4456-81e7-943fe71e09b6 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:07.923935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7aa35-348c-4456-81e7-943fe71e09b6 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:07.928681 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f40e1-f25e-44db-90f7-b010d4c436ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:07.929352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f40e1-f25e-44db-90f7-b010d4c436ad admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:07.929703 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f40e1-f25e-44db-90f7-b010d4c436ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:07.930077 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f40e1-f25e-44db-90f7-b010d4c436ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:07.930743 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f40e1-f25e-44db-90f7-b010d4c436ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:35:07.931056 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f40e1-f25e-44db-90f7-b010d4c436ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:07.932683 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f40e1-f25e-44db-90f7-b010d4c436ad admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:07.948246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 980/1988] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:35:07.953440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a05b924-e936-42da-b029-3c1aae634140 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:07.964260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a05b924-e936-42da-b029-3c1aae634140 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:07.973359 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bba385-2d0d-48d4-9034-9188c8683808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:07.974494 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bba385-2d0d-48d4-9034-9188c8683808 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:07.974847 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bba385-2d0d-48d4-9034-9188c8683808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:07.975166 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bba385-2d0d-48d4-9034-9188c8683808 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:07.976689 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bba385-2d0d-48d4-9034-9188c8683808 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:07.978917 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bba385-2d0d-48d4-9034-9188c8683808 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:08.024838 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1009/1989] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:35:07 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:08.025045 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f40e1-f25e-44db-90f7-b010d4c436ad admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:08.036372 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f40e1-f25e-44db-90f7-b010d4c436ad admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:08.110336 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da835964-0518-49df-a739-cd0646eb3a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:08.110973 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da835964-0518-49df-a739-cd0646eb3a12 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:08.111251 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da835964-0518-49df-a739-cd0646eb3a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:08.111543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da835964-0518-49df-a739-cd0646eb3a12 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:08.112238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da835964-0518-49df-a739-cd0646eb3a12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:08.114309 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da835964-0518-49df-a739-cd0646eb3a12 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:08.138892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 981/1990] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:35:08 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:08.145299 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bba385-2d0d-48d4-9034-9188c8683808 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:08.158422 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bba385-2d0d-48d4-9034-9188c8683808 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:08.166133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5f31a-83e1-4e75-84cd-9b6c3083a8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:08.167157 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5f31a-83e1-4e75-84cd-9b6c3083a8a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:08.167483 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5f31a-83e1-4e75-84cd-9b6c3083a8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:08.167759 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5f31a-83e1-4e75-84cd-9b6c3083a8a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:08.168459 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5f31a-83e1-4e75-84cd-9b6c3083a8a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:08.170195 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5f31a-83e1-4e75-84cd-9b6c3083a8a0 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:08.180808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1010/1991] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:08.185443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da835964-0518-49df-a739-cd0646eb3a12 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:08.195768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da835964-0518-49df-a739-cd0646eb3a12 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:08.201268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bb650-e91f-4657-8543-a06af943e3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:08.201984 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bb650-e91f-4657-8543-a06af943e3af admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:08.202268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bb650-e91f-4657-8543-a06af943e3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:08.202543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bb650-e91f-4657-8543-a06af943e3af admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:08.203245 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256bb650-e91f-4657-8543-a06af943e3af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:08.204916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256bb650-e91f-4657-8543-a06af943e3af admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:08.216511 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 982/1992] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:35:08.221106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5f31a-83e1-4e75-84cd-9b6c3083a8a0 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:08.231344 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5f31a-83e1-4e75-84cd-9b6c3083a8a0 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:08.237409 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea79e80-cb80-4471-839d-9146ada2a845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:08.238362 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea79e80-cb80-4471-839d-9146ada2a845 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:08.238647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea79e80-cb80-4471-839d-9146ada2a845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:08.239129 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea79e80-cb80-4471-839d-9146ada2a845 admin admin] PATH_INFO: `/v3/projects/c5b7b122124d434bba9a7d049908ab6e/users/a4d8587dfe304662b48124b952c7c8df/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:08.258618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea79e80-cb80-4471-839d-9146ada2a845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5b7b122124d434bba9a7d049908ab6e, user_id=a4d8587dfe304662b48124b952c7c8df, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:08.261469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea79e80-cb80-4471-839d-9146ada2a845 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:08.275537 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1011/1993] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:35:08 2022] PUT /identity/v3/projects/c5b7b122124d434bba9a7d049908ab6e/users/a4d8587dfe304662b48124b952c7c8df/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:08.288462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994ad1f-3557-4471-bb7d-85c2f84bc417 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:08.288745 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994ad1f-3557-4471-bb7d-85c2f84bc417 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:08.289045 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994ad1f-3557-4471-bb7d-85c2f84bc417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:08.318599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3994ad1f-3557-4471-bb7d-85c2f84bc417 None None] MFA Rules not processed for user `a4d8587dfe304662b48124b952c7c8df`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:08.320026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3994ad1f-3557-4471-bb7d-85c2f84bc417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:08.334324 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:08.393040 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 983/1994] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:08.397770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea79e80-cb80-4471-839d-9146ada2a845 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:08.463395 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea79e80-cb80-4471-839d-9146ada2a845 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:08.468081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a35d6-4050-4a85-b550-c438326eaa1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:08.468707 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a35d6-4050-4a85-b550-c438326eaa1f admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:08.468990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a35d6-4050-4a85-b550-c438326eaa1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:08.469263 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a35d6-4050-4a85-b550-c438326eaa1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:08.469964 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a35d6-4050-4a85-b550-c438326eaa1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:08.471865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a35d6-4050-4a85-b550-c438326eaa1f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:08.487888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:08.508513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1012/1995] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:35:08 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:08.509710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994ad1f-3557-4471-bb7d-85c2f84bc417 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:08.520933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994ad1f-3557-4471-bb7d-85c2f84bc417 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:08.582690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4105f5b-b1a8-44bc-aed5-b00f7fea1e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:08.583356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4105f5b-b1a8-44bc-aed5-b00f7fea1e19 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:08.583693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4105f5b-b1a8-44bc-aed5-b00f7fea1e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:08.583972 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4105f5b-b1a8-44bc-aed5-b00f7fea1e19 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:08.584683 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4105f5b-b1a8-44bc-aed5-b00f7fea1e19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:08.587440 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4105f5b-b1a8-44bc-aed5-b00f7fea1e19 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:08.615474 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 984/1996] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:35:08 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:08.616970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a35d6-4050-4a85-b550-c438326eaa1f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:08.628619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a35d6-4050-4a85-b550-c438326eaa1f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:08.633435 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b17ccd-dae7-445c-82ef-777f89336258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:08.634101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b17ccd-dae7-445c-82ef-777f89336258 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:08.634382 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b17ccd-dae7-445c-82ef-777f89336258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:08.634653 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b17ccd-dae7-445c-82ef-777f89336258 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:08.635352 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b17ccd-dae7-445c-82ef-777f89336258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:08.637135 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b17ccd-dae7-445c-82ef-777f89336258 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:08.649994 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1013/1997] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:08.653086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4105f5b-b1a8-44bc-aed5-b00f7fea1e19 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:08.663885 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4105f5b-b1a8-44bc-aed5-b00f7fea1e19 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:08.668615 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8435f1-64e3-489f-96f1-b990799974b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:08.669309 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8435f1-64e3-489f-96f1-b990799974b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:08.669593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8435f1-64e3-489f-96f1-b990799974b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:08.669872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8435f1-64e3-489f-96f1-b990799974b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:08.671147 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8435f1-64e3-489f-96f1-b990799974b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:08.672216 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8435f1-64e3-489f-96f1-b990799974b0 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:08.683032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 985/1998] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:08.688473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b17ccd-dae7-445c-82ef-777f89336258 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:08.700189 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b17ccd-dae7-445c-82ef-777f89336258 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:08.705573 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330300e-3257-4657-ab7f-558c5f089652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:08.706290 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330300e-3257-4657-ab7f-558c5f089652 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:08.706571 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330300e-3257-4657-ab7f-558c5f089652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:08.706847 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330300e-3257-4657-ab7f-558c5f089652 admin admin] PATH_INFO: `/v3/projects/7a1846cc9abe4b56af27b7892d4d4c50/users/000d3f63632447799a9b461f06388be9/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:08.723867 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d330300e-3257-4657-ab7f-558c5f089652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1846cc9abe4b56af27b7892d4d4c50, user_id=000d3f63632447799a9b461f06388be9, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:08.726636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d330300e-3257-4657-ab7f-558c5f089652 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:08.742626 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1014/1999] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:35:08 2022] PUT /identity/v3/projects/7a1846cc9abe4b56af27b7892d4d4c50/users/000d3f63632447799a9b461f06388be9/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:08.756846 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a92baf-2b25-4df1-b3ab-1b68266e1406 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:08.757315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a92baf-2b25-4df1-b3ab-1b68266e1406 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:08.757752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a92baf-2b25-4df1-b3ab-1b68266e1406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:08.787485 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c4a92baf-2b25-4df1-b3ab-1b68266e1406 None None] MFA Rules not processed for user `000d3f63632447799a9b461f06388be9`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:08.788754 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c4a92baf-2b25-4df1-b3ab-1b68266e1406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:08.864462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 986/2000] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:08.869281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330300e-3257-4657-ab7f-558c5f089652 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:08.941404 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330300e-3257-4657-ab7f-558c5f089652 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:08.947250 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e96ef-e933-4aa5-b0b2-0682bb44b8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:08.947917 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e96ef-e933-4aa5-b0b2-0682bb44b8a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:08.948203 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e96ef-e933-4aa5-b0b2-0682bb44b8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:08.948478 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e96ef-e933-4aa5-b0b2-0682bb44b8a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:08.949177 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e96ef-e933-4aa5-b0b2-0682bb44b8a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:08.951737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e96ef-e933-4aa5-b0b2-0682bb44b8a0 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:08.981865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1015/2001] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:35:08 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:08.985991 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a92baf-2b25-4df1-b3ab-1b68266e1406 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:08.998557 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a92baf-2b25-4df1-b3ab-1b68266e1406 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:09.022963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:09.087408 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf6509-75d3-4b34-ac4b-e706812f0638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:09.088066 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf6509-75d3-4b34-ac4b-e706812f0638 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:09.088347 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf6509-75d3-4b34-ac4b-e706812f0638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:09.088719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf6509-75d3-4b34-ac4b-e706812f0638 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:09.089415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf6509-75d3-4b34-ac4b-e706812f0638 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:09.091857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf6509-75d3-4b34-ac4b-e706812f0638 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:09.118400 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 987/2002] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:35:08 2022] POST /identity/v3/users => generated 295 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:09.122182 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e96ef-e933-4aa5-b0b2-0682bb44b8a0 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:09.139906 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e96ef-e933-4aa5-b0b2-0682bb44b8a0 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:09.145288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab60df-6ff5-4917-bee0-fcfe9b2b3555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:09.146053 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab60df-6ff5-4917-bee0-fcfe9b2b3555 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:09.146341 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab60df-6ff5-4917-bee0-fcfe9b2b3555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:09.146622 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab60df-6ff5-4917-bee0-fcfe9b2b3555 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:09.150436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab60df-6ff5-4917-bee0-fcfe9b2b3555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:09.152239 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab60df-6ff5-4917-bee0-fcfe9b2b3555 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:09.164821 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1016/2003] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:09.169112 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf6509-75d3-4b34-ac4b-e706812f0638 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:09.180451 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf6509-75d3-4b34-ac4b-e706812f0638 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:09.186013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2a683-9ece-40ae-9393-f1f48836147b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:09.186866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2a683-9ece-40ae-9393-f1f48836147b admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:09.187159 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2a683-9ece-40ae-9393-f1f48836147b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:09.187437 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2a683-9ece-40ae-9393-f1f48836147b admin admin] PATH_INFO: `/v3/projects/c3b156ec7a21410098eed71fc805e76c/users/f3b0c0a23a124a4395494b734a67e63e/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:09.209094 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2a683-9ece-40ae-9393-f1f48836147b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3b156ec7a21410098eed71fc805e76c, user_id=f3b0c0a23a124a4395494b734a67e63e, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:09.212159 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2a683-9ece-40ae-9393-f1f48836147b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:09.227642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 988/2004] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:35:09 2022] PUT /identity/v3/projects/c3b156ec7a21410098eed71fc805e76c/users/f3b0c0a23a124a4395494b734a67e63e/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:09.233833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab60df-6ff5-4917-bee0-fcfe9b2b3555 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:09.246144 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab60df-6ff5-4917-bee0-fcfe9b2b3555 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:09.316087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca6721-9461-4c6d-a9ba-45e0047d1432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:09.316733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca6721-9461-4c6d-a9ba-45e0047d1432 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:09.317014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca6721-9461-4c6d-a9ba-45e0047d1432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:09.317295 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca6721-9461-4c6d-a9ba-45e0047d1432 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:09.318096 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca6721-9461-4c6d-a9ba-45e0047d1432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:09.319738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca6721-9461-4c6d-a9ba-45e0047d1432 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:09.334266 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1017/2005] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:09.335887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2a683-9ece-40ae-9393-f1f48836147b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:09.350228 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2a683-9ece-40ae-9393-f1f48836147b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:09.355633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2899f6-b616-4c50-a844-2147804b72e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:09.356447 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2899f6-b616-4c50-a844-2147804b72e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:09.357328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2899f6-b616-4c50-a844-2147804b72e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:09.357674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2899f6-b616-4c50-a844-2147804b72e1 admin admin] PATH_INFO: `/v3/projects/c3b156ec7a21410098eed71fc805e76c/users/f3b0c0a23a124a4395494b734a67e63e/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:09.360006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2899f6-b616-4c50-a844-2147804b72e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3b156ec7a21410098eed71fc805e76c, user_id=f3b0c0a23a124a4395494b734a67e63e, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:09.362997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2899f6-b616-4c50-a844-2147804b72e1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:09.376265 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 989/2006] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:35:09 2022] PUT /identity/v3/projects/c3b156ec7a21410098eed71fc805e76c/users/f3b0c0a23a124a4395494b734a67e63e/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:09.386982 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992059e5-e957-4631-9fa1-887b341c98a8 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:09.387271 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992059e5-e957-4631-9fa1-887b341c98a8 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:09.387551 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992059e5-e957-4631-9fa1-887b341c98a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:09.438304 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-992059e5-e957-4631-9fa1-887b341c98a8 None None] MFA Rules not processed for user `f3b0c0a23a124a4395494b734a67e63e`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:09.439633 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-992059e5-e957-4631-9fa1-887b341c98a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:09.463235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:09.519085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1018/2007] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:09.530739 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2899f6-b616-4c50-a844-2147804b72e1 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:09.532607 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-aa2899f6-b616-4c50-a844-2147804b72e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:09.598812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2899f6-b616-4c50-a844-2147804b72e1 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:09.602709 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405b1e6-8c98-47bb-a5c9-65a752352465 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:09.603354 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405b1e6-8c98-47bb-a5c9-65a752352465 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:09.603640 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405b1e6-8c98-47bb-a5c9-65a752352465 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:09.603931 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405b1e6-8c98-47bb-a5c9-65a752352465 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:09.664448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9405b1e6-8c98-47bb-a5c9-65a752352465 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:09.665944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9405b1e6-8c98-47bb-a5c9-65a752352465 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:09.675512 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 990/2008] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:35:11.352602 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992059e5-e957-4631-9fa1-887b341c98a8 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:11.354085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-992059e5-e957-4631-9fa1-887b341c98a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:11.368559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992059e5-e957-4631-9fa1-887b341c98a8 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:11.372449 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faffa68-fcce-409c-b38f-018f180e3a74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:11.373088 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faffa68-fcce-409c-b38f-018f180e3a74 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:11.373374 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faffa68-fcce-409c-b38f-018f180e3a74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:11.373655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faffa68-fcce-409c-b38f-018f180e3a74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:11.443402 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faffa68-fcce-409c-b38f-018f180e3a74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:11.444854 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faffa68-fcce-409c-b38f-018f180e3a74 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:11.455390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1019/2009] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:35:12.080811 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405b1e6-8c98-47bb-a5c9-65a752352465 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:12.094320 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405b1e6-8c98-47bb-a5c9-65a752352465 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:12.098633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31a25b-7348-4c01-9773-cc8b98c483bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:12.099277 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31a25b-7348-4c01-9773-cc8b98c483bc service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:12.099634 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31a25b-7348-4c01-9773-cc8b98c483bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:12.099920 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31a25b-7348-4c01-9773-cc8b98c483bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:12.112829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31a25b-7348-4c01-9773-cc8b98c483bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:12.114463 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31a25b-7348-4c01-9773-cc8b98c483bc service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:12.130160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 991/2010] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:35:16.160573 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faffa68-fcce-409c-b38f-018f180e3a74 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:16.269989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faffa68-fcce-409c-b38f-018f180e3a74 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:16.278143 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3bebc-9007-4908-bbde-98c413e352ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:16.278143 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3bebc-9007-4908-bbde-98c413e352ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:16.278143 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3bebc-9007-4908-bbde-98c413e352ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:16.278143 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3bebc-9007-4908-bbde-98c413e352ff admin admin] PATH_INFO: `/v3/users/a4d8587dfe304662b48124b952c7c8df` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:16.278476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3bebc-9007-4908-bbde-98c413e352ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4d8587dfe304662b48124b952c7c8df)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:16.279832 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3bebc-9007-4908-bbde-98c413e352ff admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:16.353458 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b1b3bebc-9007-4908-bbde-98c413e352ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4d8587dfe304662b48124b952c7c8df'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:16.354224 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b1b3bebc-9007-4908-bbde-98c413e352ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4d8587dfe304662b48124b952c7c8df'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:16.361999 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b1b3bebc-9007-4908-bbde-98c413e352ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4d8587dfe304662b48124b952c7c8df'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:16.364485 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:16.378112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b1b3bebc-9007-4908-bbde-98c413e352ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4d8587dfe304662b48124b952c7c8df'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:16.385230 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b1b3bebc-9007-4908-bbde-98c413e352ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4d8587dfe304662b48124b952c7c8df'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:16.394967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1020/2011] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:35:16 2022] DELETE /identity/v3/users/a4d8587dfe304662b48124b952c7c8df => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:16.396618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dd40a-a960-45e3-a8aa-b8ecefd015d3 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:16.397071 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dd40a-a960-45e3-a8aa-b8ecefd015d3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:16.397188 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dd40a-a960-45e3-a8aa-b8ecefd015d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:16.425245 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c63dd40a-a960-45e3-a8aa-b8ecefd015d3 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:16.426528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c63dd40a-a960-45e3-a8aa-b8ecefd015d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:16.440503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:16.496057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 992/2012] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:16.507236 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3bebc-9007-4908-bbde-98c413e352ff admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:16.508979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b1b3bebc-9007-4908-bbde-98c413e352ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:16.608488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3bebc-9007-4908-bbde-98c413e352ff admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:16.612568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb695407-60d8-4527-bcf4-a93e3f1be653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:16.613213 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb695407-60d8-4527-bcf4-a93e3f1be653 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:16.613497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb695407-60d8-4527-bcf4-a93e3f1be653 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:16.614919 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb695407-60d8-4527-bcf4-a93e3f1be653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:16.624750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb695407-60d8-4527-bcf4-a93e3f1be653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:16.626459 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb695407-60d8-4527-bcf4-a93e3f1be653 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:16.642767 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1021/2013] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:35:17.221490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63dd40a-a960-45e3-a8aa-b8ecefd015d3 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:17.223510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c63dd40a-a960-45e3-a8aa-b8ecefd015d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:17.243246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63dd40a-a960-45e3-a8aa-b8ecefd015d3 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:17.254630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89b0fb-a17a-49dd-b1a1-60fcab298bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:17.255315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89b0fb-a17a-49dd-b1a1-60fcab298bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:17.255605 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89b0fb-a17a-49dd-b1a1-60fcab298bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:17.255890 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89b0fb-a17a-49dd-b1a1-60fcab298bbd admin admin] PATH_INFO: `/v3/projects/c5b7b122124d434bba9a7d049908ab6e` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:17.260119 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89b0fb-a17a-49dd-b1a1-60fcab298bbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5b7b122124d434bba9a7d049908ab6e)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:17.262244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89b0fb-a17a-49dd-b1a1-60fcab298bbd admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:17.318983 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5c89b0fb-a17a-49dd-b1a1-60fcab298bbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5b7b122124d434bba9a7d049908ab6e', 'request_id': 'req-5c89b0fb-a17a-49dd-b1a1-60fcab298bbd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:17.326011 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5c89b0fb-a17a-49dd-b1a1-60fcab298bbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5b7b122124d434bba9a7d049908ab6e', 'request_id': 'req-5c89b0fb-a17a-49dd-b1a1-60fcab298bbd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:17.335242 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5c89b0fb-a17a-49dd-b1a1-60fcab298bbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5b7b122124d434bba9a7d049908ab6e', 'request_id': 'req-5c89b0fb-a17a-49dd-b1a1-60fcab298bbd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:17.351644 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5c89b0fb-a17a-49dd-b1a1-60fcab298bbd admin admin] The token cache is being invalidate because project c5b7b122124d434bba9a7d049908ab6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:35:17.351941 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5c89b0fb-a17a-49dd-b1a1-60fcab298bbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:17.352614 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5c89b0fb-a17a-49dd-b1a1-60fcab298bbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:17.354249 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 993/2014] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:35:17 2022] DELETE /identity/v3/projects/c5b7b122124d434bba9a7d049908ab6e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:17.358587 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb695407-60d8-4527-bcf4-a93e3f1be653 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:17.368736 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb695407-60d8-4527-bcf4-a93e3f1be653 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:17.431952 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d301f7-9f86-4133-ad5d-08042def3a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:17.433015 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d301f7-9f86-4133-ad5d-08042def3a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:17.433452 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d301f7-9f86-4133-ad5d-08042def3a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:17.434082 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d301f7-9f86-4133-ad5d-08042def3a2b admin admin] PATH_INFO: `/v3/users/000d3f63632447799a9b461f06388be9` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:17.436329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d301f7-9f86-4133-ad5d-08042def3a2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=000d3f63632447799a9b461f06388be9)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:17.439313 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d301f7-9f86-4133-ad5d-08042def3a2b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:17.523841 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-87d301f7-9f86-4133-ad5d-08042def3a2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '000d3f63632447799a9b461f06388be9'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:17.526033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-87d301f7-9f86-4133-ad5d-08042def3a2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '000d3f63632447799a9b461f06388be9'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:17.538190 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-87d301f7-9f86-4133-ad5d-08042def3a2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '000d3f63632447799a9b461f06388be9'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:17.565680 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-87d301f7-9f86-4133-ad5d-08042def3a2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '000d3f63632447799a9b461f06388be9'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:17.579218 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:17.581590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-87d301f7-9f86-4133-ad5d-08042def3a2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '000d3f63632447799a9b461f06388be9'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:17.600523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1022/2015] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:35:17 2022] DELETE /identity/v3/users/000d3f63632447799a9b461f06388be9 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:35:17.969080 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89b0fb-a17a-49dd-b1a1-60fcab298bbd admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:17.970611 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5c89b0fb-a17a-49dd-b1a1-60fcab298bbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:17.984216 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89b0fb-a17a-49dd-b1a1-60fcab298bbd admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:18.088260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d088158e-5d1c-4cea-a559-ff7e81ab7c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:18.088927 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d088158e-5d1c-4cea-a559-ff7e81ab7c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:18.089214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d088158e-5d1c-4cea-a559-ff7e81ab7c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:18.089494 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d088158e-5d1c-4cea-a559-ff7e81ab7c41 admin admin] PATH_INFO: `/v3/projects/7a1846cc9abe4b56af27b7892d4d4c50` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:18.090741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d088158e-5d1c-4cea-a559-ff7e81ab7c41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a1846cc9abe4b56af27b7892d4d4c50)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:18.093223 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d088158e-5d1c-4cea-a559-ff7e81ab7c41 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:18.143751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:18.148886 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d088158e-5d1c-4cea-a559-ff7e81ab7c41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a1846cc9abe4b56af27b7892d4d4c50', 'request_id': 'req-d088158e-5d1c-4cea-a559-ff7e81ab7c41'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:18.158094 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d088158e-5d1c-4cea-a559-ff7e81ab7c41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a1846cc9abe4b56af27b7892d4d4c50', 'request_id': 'req-d088158e-5d1c-4cea-a559-ff7e81ab7c41'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:18.164611 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d088158e-5d1c-4cea-a559-ff7e81ab7c41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a1846cc9abe4b56af27b7892d4d4c50', 'request_id': 'req-d088158e-5d1c-4cea-a559-ff7e81ab7c41'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:18.179502 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d088158e-5d1c-4cea-a559-ff7e81ab7c41 admin admin] The token cache is being invalidate because project 7a1846cc9abe4b56af27b7892d4d4c50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:35:18.179994 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d088158e-5d1c-4cea-a559-ff7e81ab7c41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:18.187243 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d088158e-5d1c-4cea-a559-ff7e81ab7c41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:18.193035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 994/2016] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:35:17 2022] DELETE /identity/v3/projects/7a1846cc9abe4b56af27b7892d4d4c50 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:18.193732 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d301f7-9f86-4133-ad5d-08042def3a2b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:18.195213 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-87d301f7-9f86-4133-ad5d-08042def3a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:18.209130 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d301f7-9f86-4133-ad5d-08042def3a2b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:18.277814 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad600485-5c3d-4037-a587-57311d265cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:18.278819 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad600485-5c3d-4037-a587-57311d265cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:18.279112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad600485-5c3d-4037-a587-57311d265cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:18.279441 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad600485-5c3d-4037-a587-57311d265cb6 admin admin] PATH_INFO: `/v3/users/f3b0c0a23a124a4395494b734a67e63e` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:18.280688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad600485-5c3d-4037-a587-57311d265cb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3b0c0a23a124a4395494b734a67e63e)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:18.282761 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad600485-5c3d-4037-a587-57311d265cb6 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:18.343711 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ad600485-5c3d-4037-a587-57311d265cb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3b0c0a23a124a4395494b734a67e63e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:18.344392 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ad600485-5c3d-4037-a587-57311d265cb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3b0c0a23a124a4395494b734a67e63e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:18.354139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ad600485-5c3d-4037-a587-57311d265cb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3b0c0a23a124a4395494b734a67e63e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:18.368296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ad600485-5c3d-4037-a587-57311d265cb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3b0c0a23a124a4395494b734a67e63e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:18.378958 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ad600485-5c3d-4037-a587-57311d265cb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3b0c0a23a124a4395494b734a67e63e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:18.385864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1023/2017] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:35:18 2022] DELETE /identity/v3/users/f3b0c0a23a124a4395494b734a67e63e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:35:18.921080 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d088158e-5d1c-4cea-a559-ff7e81ab7c41 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:18.923046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d088158e-5d1c-4cea-a559-ff7e81ab7c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:18.948450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d088158e-5d1c-4cea-a559-ff7e81ab7c41 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:19.019079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b1928-e4ac-4568-bbc1-13ead10bc14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:19.019914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b1928-e4ac-4568-bbc1-13ead10bc14d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:19.020007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b1928-e4ac-4568-bbc1-13ead10bc14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:19.020290 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b1928-e4ac-4568-bbc1-13ead10bc14d admin admin] PATH_INFO: `/v3/projects/c3b156ec7a21410098eed71fc805e76c` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:19.021499 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b1928-e4ac-4568-bbc1-13ead10bc14d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3b156ec7a21410098eed71fc805e76c)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:19.023531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b1928-e4ac-4568-bbc1-13ead10bc14d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:19.085741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-828b1928-e4ac-4568-bbc1-13ead10bc14d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3b156ec7a21410098eed71fc805e76c', 'request_id': 'req-828b1928-e4ac-4568-bbc1-13ead10bc14d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:19.094130 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-828b1928-e4ac-4568-bbc1-13ead10bc14d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3b156ec7a21410098eed71fc805e76c', 'request_id': 'req-828b1928-e4ac-4568-bbc1-13ead10bc14d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:19.102015 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-828b1928-e4ac-4568-bbc1-13ead10bc14d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3b156ec7a21410098eed71fc805e76c', 'request_id': 'req-828b1928-e4ac-4568-bbc1-13ead10bc14d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:19.118309 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-828b1928-e4ac-4568-bbc1-13ead10bc14d admin admin] The token cache is being invalidate because project c3b156ec7a21410098eed71fc805e76c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:35:19.118508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-828b1928-e4ac-4568-bbc1-13ead10bc14d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:19.119144 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-828b1928-e4ac-4568-bbc1-13ead10bc14d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:19.120849 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 995/2018] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:35:18 2022] DELETE /identity/v3/projects/c3b156ec7a21410098eed71fc805e76c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:19.129415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50ce2c-08ca-43b8-90d4-ae655b07efc1 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:19.129415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50ce2c-08ca-43b8-90d4-ae655b07efc1 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:19.129415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50ce2c-08ca-43b8-90d4-ae655b07efc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:19.158573 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-3a50ce2c-08ca-43b8-90d4-ae655b07efc1 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:19.160833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3a50ce2c-08ca-43b8-90d4-ae655b07efc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:19.187985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b1928-e4ac-4568-bbc1-13ead10bc14d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:19.189864 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-828b1928-e4ac-4568-bbc1-13ead10bc14d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:19.251590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1024/2019] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:19.260385 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa3071-d65a-426d-84ef-cc4962d78b39 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:19.260669 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa3071-d65a-426d-84ef-cc4962d78b39 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:19.260942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa3071-d65a-426d-84ef-cc4962d78b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:19.288661 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b1928-e4ac-4568-bbc1-13ead10bc14d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:19.291001 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-3eaa3071-d65a-426d-84ef-cc4962d78b39 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:19.292344 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3eaa3071-d65a-426d-84ef-cc4962d78b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:19.294331 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e9a51-b847-4f50-9614-fbad2b814176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:19.295451 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e9a51-b847-4f50-9614-fbad2b814176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:19.295451 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e9a51-b847-4f50-9614-fbad2b814176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:19.295736 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e9a51-b847-4f50-9614-fbad2b814176 admin admin] PATH_INFO: `/v3/users/c07d9c30b18a4150aa0bc7a1705551d7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:19.296883 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e9a51-b847-4f50-9614-fbad2b814176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c07d9c30b18a4150aa0bc7a1705551d7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:19.298895 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e9a51-b847-4f50-9614-fbad2b814176 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:19.301910 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:19.306004 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1025/2020] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:19.311972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa3071-d65a-426d-84ef-cc4962d78b39 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:19.330098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa3071-d65a-426d-84ef-cc4962d78b39 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:19.347663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b582af-bc11-4544-b287-04d76c625418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:19.348457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b582af-bc11-4544-b287-04d76c625418 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:19.348791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b582af-bc11-4544-b287-04d76c625418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:19.349052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b582af-bc11-4544-b287-04d76c625418 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:19.349835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b582af-bc11-4544-b287-04d76c625418 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:35:19.350204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b582af-bc11-4544-b287-04d76c625418 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:19.354591 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b582af-bc11-4544-b287-04d76c625418 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:19.364058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:19.379889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1026/2021] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:35:19.382014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b582af-bc11-4544-b287-04d76c625418 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:19.389624 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-120e9a51-b847-4f50-9614-fbad2b814176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c07d9c30b18a4150aa0bc7a1705551d7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:19.390499 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-120e9a51-b847-4f50-9614-fbad2b814176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c07d9c30b18a4150aa0bc7a1705551d7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:19.396327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b582af-bc11-4544-b287-04d76c625418 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:19.402059 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e02b31-c059-44dc-9649-7765ad89ae9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:19.402693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e02b31-c059-44dc-9649-7765ad89ae9a admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:19.402827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-120e9a51-b847-4f50-9614-fbad2b814176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c07d9c30b18a4150aa0bc7a1705551d7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:19.403030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e02b31-c059-44dc-9649-7765ad89ae9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:19.403568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e02b31-c059-44dc-9649-7765ad89ae9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:19.404340 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e02b31-c059-44dc-9649-7765ad89ae9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:19.406215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e02b31-c059-44dc-9649-7765ad89ae9a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:19.422380 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-120e9a51-b847-4f50-9614-fbad2b814176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c07d9c30b18a4150aa0bc7a1705551d7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:19.433854 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-120e9a51-b847-4f50-9614-fbad2b814176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c07d9c30b18a4150aa0bc7a1705551d7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:19.439113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1027/2022] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:35:19 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:19.443829 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e02b31-c059-44dc-9649-7765ad89ae9a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:19.447526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-39e02b31-c059-44dc-9649-7765ad89ae9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:19.461264 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 996/2023] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:35:19 2022] DELETE /identity/v3/users/c07d9c30b18a4150aa0bc7a1705551d7 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:35:19.550452 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e02b31-c059-44dc-9649-7765ad89ae9a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:19.615491 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2deb9bd-1dac-4311-ba3b-6396b51e2aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:19.615982 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2deb9bd-1dac-4311-ba3b-6396b51e2aea admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:19.616273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2deb9bd-1dac-4311-ba3b-6396b51e2aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:19.616838 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2deb9bd-1dac-4311-ba3b-6396b51e2aea admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:19.617266 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2deb9bd-1dac-4311-ba3b-6396b51e2aea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:19.619372 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2deb9bd-1dac-4311-ba3b-6396b51e2aea admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:19.647873 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1028/2024] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:35:19 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:19.652726 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e9a51-b847-4f50-9614-fbad2b814176 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:19.667450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e9a51-b847-4f50-9614-fbad2b814176 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:19.672786 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d877f-2b52-4f0f-80d7-00054fc48ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:19.673455 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d877f-2b52-4f0f-80d7-00054fc48ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:19.673773 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d877f-2b52-4f0f-80d7-00054fc48ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:19.674041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d877f-2b52-4f0f-80d7-00054fc48ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:19.674752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d877f-2b52-4f0f-80d7-00054fc48ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:19.676418 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d877f-2b52-4f0f-80d7-00054fc48ef9 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:19.678558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2deb9bd-1dac-4311-ba3b-6396b51e2aea admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:19.680460 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c2deb9bd-1dac-4311-ba3b-6396b51e2aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:19.698094 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2deb9bd-1dac-4311-ba3b-6396b51e2aea admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:19.699444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 997/2025] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:19.704348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d877f-2b52-4f0f-80d7-00054fc48ef9 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:19.706380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9794c77-e00d-413e-ae0b-45069f74b952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:19.707237 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9794c77-e00d-413e-ae0b-45069f74b952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:19.707733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9794c77-e00d-413e-ae0b-45069f74b952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:19.709756 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9794c77-e00d-413e-ae0b-45069f74b952 admin admin] PATH_INFO: `/v3/projects/db84eff445df4738a5fccb348fe9afeb` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:19.711985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9794c77-e00d-413e-ae0b-45069f74b952 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db84eff445df4738a5fccb348fe9afeb)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:19.714792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9794c77-e00d-413e-ae0b-45069f74b952 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:19.733337 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d877f-2b52-4f0f-80d7-00054fc48ef9 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:19.738471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54803f49-b746-4495-85de-76b2336e9ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:19.739314 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54803f49-b746-4495-85de-76b2336e9ede admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:19.739770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54803f49-b746-4495-85de-76b2336e9ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:19.740214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54803f49-b746-4495-85de-76b2336e9ede admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:19.741217 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54803f49-b746-4495-85de-76b2336e9ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:19.743795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54803f49-b746-4495-85de-76b2336e9ede admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:19.757990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 998/2026] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:19.762386 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54803f49-b746-4495-85de-76b2336e9ede admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:19.775238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54803f49-b746-4495-85de-76b2336e9ede admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:19.803688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f9794c77-e00d-413e-ae0b-45069f74b952 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db84eff445df4738a5fccb348fe9afeb', 'request_id': 'req-f9794c77-e00d-413e-ae0b-45069f74b952'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:19.813272 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f9794c77-e00d-413e-ae0b-45069f74b952 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db84eff445df4738a5fccb348fe9afeb', 'request_id': 'req-f9794c77-e00d-413e-ae0b-45069f74b952'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:19.840929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f9794c77-e00d-413e-ae0b-45069f74b952 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db84eff445df4738a5fccb348fe9afeb', 'request_id': 'req-f9794c77-e00d-413e-ae0b-45069f74b952'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:19.844861 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:19.866593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f9794c77-e00d-413e-ae0b-45069f74b952 admin admin] The token cache is being invalidate because project db84eff445df4738a5fccb348fe9afeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:35:19.867027 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f9794c77-e00d-413e-ae0b-45069f74b952 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:19.867661 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f9794c77-e00d-413e-ae0b-45069f74b952 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:19.870594 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1029/2027] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:35:19 2022] DELETE /identity/v3/projects/db84eff445df4738a5fccb348fe9afeb => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:19.875386 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a32f12-42f9-49f7-84ee-8779d4d96427 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:19.875674 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a32f12-42f9-49f7-84ee-8779d4d96427 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:19.875946 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a32f12-42f9-49f7-84ee-8779d4d96427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:19.880381 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a304c57-1209-45e9-b907-410ee9b547ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:19.881272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a304c57-1209-45e9-b907-410ee9b547ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:19.881733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a304c57-1209-45e9-b907-410ee9b547ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:19.882300 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a304c57-1209-45e9-b907-410ee9b547ee admin admin] PATH_INFO: `/v3/projects/38f80d210cce4a41af527ac134deba93/users/77cb30fae1184064a8c4f822d4d31096/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:19.899730 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f0a32f12-42f9-49f7-84ee-8779d4d96427 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:19.901091 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f0a32f12-42f9-49f7-84ee-8779d4d96427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:19.906230 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a304c57-1209-45e9-b907-410ee9b547ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=38f80d210cce4a41af527ac134deba93, user_id=77cb30fae1184064a8c4f822d4d31096, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:19.909209 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a304c57-1209-45e9-b907-410ee9b547ee admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:19.911046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1030/2028] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:19.920383 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615f8e4-b65c-4a47-91b2-c62fbdd4ba2e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:19.920705 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615f8e4-b65c-4a47-91b2-c62fbdd4ba2e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:19.922433 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615f8e4-b65c-4a47-91b2-c62fbdd4ba2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:19.936556 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 999/2029] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:35:19 2022] PUT /identity/v3/projects/38f80d210cce4a41af527ac134deba93/users/77cb30fae1184064a8c4f822d4d31096/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:19.942023 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43d352-03be-4e15-b05e-2ea391d916aa None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:19.942417 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43d352-03be-4e15-b05e-2ea391d916aa None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:19.942980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43d352-03be-4e15-b05e-2ea391d916aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:19.944416 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1615f8e4-b65c-4a47-91b2-c62fbdd4ba2e None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:19.946813 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1615f8e4-b65c-4a47-91b2-c62fbdd4ba2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:19.973073 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-fd43d352-03be-4e15-b05e-2ea391d916aa None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:19.974490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-fd43d352-03be-4e15-b05e-2ea391d916aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:20.032565 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1031/2030] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:35:20.037171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1615f8e4-b65c-4a47-91b2-c62fbdd4ba2e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:20.047835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1615f8e4-b65c-4a47-91b2-c62fbdd4ba2e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:20.048028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:20.058465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07358d0f-668c-427f-b640-1b1d77376d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:20.059196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07358d0f-668c-427f-b640-1b1d77376d44 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:20.059549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07358d0f-668c-427f-b640-1b1d77376d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:20.059939 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07358d0f-668c-427f-b640-1b1d77376d44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:20.065816 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07358d0f-668c-427f-b640-1b1d77376d44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:35:20.065816 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07358d0f-668c-427f-b640-1b1d77376d44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:20.065816 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07358d0f-668c-427f-b640-1b1d77376d44 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:20.076224 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1000/2031] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:35:20.077992 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1032/2032] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:35:20.082778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43d352-03be-4e15-b05e-2ea391d916aa None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:20.087120 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07358d0f-668c-427f-b640-1b1d77376d44 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:20.088978 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-07358d0f-668c-427f-b640-1b1d77376d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:20.093876 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43d352-03be-4e15-b05e-2ea391d916aa None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:20.102874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89890b9-5ce3-480f-a49f-ef5d9cb5c2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:20.103758 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89890b9-5ce3-480f-a49f-ef5d9cb5c2da admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:20.104235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89890b9-5ce3-480f-a49f-ef5d9cb5c2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:20.104512 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89890b9-5ce3-480f-a49f-ef5d9cb5c2da admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:20.105263 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89890b9-5ce3-480f-a49f-ef5d9cb5c2da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:20.107303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89890b9-5ce3-480f-a49f-ef5d9cb5c2da admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:20.137750 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1001/2033] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:35:20 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:20.143218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89890b9-5ce3-480f-a49f-ef5d9cb5c2da admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:20.169678 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07358d0f-668c-427f-b640-1b1d77376d44 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:20.207101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89890b9-5ce3-480f-a49f-ef5d9cb5c2da admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:20.213363 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750eaf0-976a-4702-b228-2379bc309d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:20.213937 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750eaf0-976a-4702-b228-2379bc309d56 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:20.214118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750eaf0-976a-4702-b228-2379bc309d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:20.214389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750eaf0-976a-4702-b228-2379bc309d56 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:20.215075 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750eaf0-976a-4702-b228-2379bc309d56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:20.217059 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750eaf0-976a-4702-b228-2379bc309d56 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:20.221255 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbc6c3-27bc-4fed-942a-71f08115628d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:20.221905 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbc6c3-27bc-4fed-942a-71f08115628d service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:20.222188 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbc6c3-27bc-4fed-942a-71f08115628d service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:20.222460 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbc6c3-27bc-4fed-942a-71f08115628d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:20.232921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fbc6c3-27bc-4fed-942a-71f08115628d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:20.234504 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fbc6c3-27bc-4fed-942a-71f08115628d service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:20.240414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1002/2034] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:35:20 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:35:20.244899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750eaf0-976a-4702-b228-2379bc309d56 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:20.245406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1033/2035] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:35:20.256379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750eaf0-976a-4702-b228-2379bc309d56 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:20.261501 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25087860-686a-4c6d-b6c6-e4246a7f5c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:20.262163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25087860-686a-4c6d-b6c6-e4246a7f5c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:20.262446 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25087860-686a-4c6d-b6c6-e4246a7f5c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:20.262719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25087860-686a-4c6d-b6c6-e4246a7f5c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:20.264022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25087860-686a-4c6d-b6c6-e4246a7f5c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:20.265950 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25087860-686a-4c6d-b6c6-e4246a7f5c2a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:20.277629 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1003/2036] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:35:20.281668 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbc6c3-27bc-4fed-942a-71f08115628d service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:20.291536 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbc6c3-27bc-4fed-942a-71f08115628d service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:20.296167 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c311853-4924-485c-84f4-24ded9d8e10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:20.296779 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c311853-4924-485c-84f4-24ded9d8e10b admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:20.297060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c311853-4924-485c-84f4-24ded9d8e10b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:20.297329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c311853-4924-485c-84f4-24ded9d8e10b admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:20.298065 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c311853-4924-485c-84f4-24ded9d8e10b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:20.299654 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c311853-4924-485c-84f4-24ded9d8e10b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:20.311220 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1034/2037] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:20.315174 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25087860-686a-4c6d-b6c6-e4246a7f5c2a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:20.328935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25087860-686a-4c6d-b6c6-e4246a7f5c2a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:20.334059 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d16748-15b7-47f2-bb4a-218a5d571a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:20.334752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d16748-15b7-47f2-bb4a-218a5d571a3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:20.335337 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d16748-15b7-47f2-bb4a-218a5d571a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:20.335691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d16748-15b7-47f2-bb4a-218a5d571a3b admin admin] PATH_INFO: `/v3/projects/53f7598bdaaf4898acfa6988952d1b91/users/a1fe60c8967544c0b849ec9a89c9afbb/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:20.352847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d16748-15b7-47f2-bb4a-218a5d571a3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=53f7598bdaaf4898acfa6988952d1b91, user_id=a1fe60c8967544c0b849ec9a89c9afbb, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:20.355622 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d16748-15b7-47f2-bb4a-218a5d571a3b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:20.370939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1004/2038] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:35:20 2022] PUT /identity/v3/projects/53f7598bdaaf4898acfa6988952d1b91/users/a1fe60c8967544c0b849ec9a89c9afbb/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:20.383368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec2ec1-24f3-46d7-bf84-b8f3cb98d304 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:20.383811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec2ec1-24f3-46d7-bf84-b8f3cb98d304 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:20.383811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec2ec1-24f3-46d7-bf84-b8f3cb98d304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:20.414774 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-3bec2ec1-24f3-46d7-bf84-b8f3cb98d304 None None] MFA Rules not processed for user `a1fe60c8967544c0b849ec9a89c9afbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:20.416003 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3bec2ec1-24f3-46d7-bf84-b8f3cb98d304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:20.426158 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:20.493943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1035/2039] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:35:20.506058 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d16748-15b7-47f2-bb4a-218a5d571a3b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:20.507885 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b1d16748-15b7-47f2-bb4a-218a5d571a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:20.613958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d16748-15b7-47f2-bb4a-218a5d571a3b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:20.623136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025e11f-4fa0-4291-a5c1-fadd4abbca40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:20.624056 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025e11f-4fa0-4291-a5c1-fadd4abbca40 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:20.624401 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025e11f-4fa0-4291-a5c1-fadd4abbca40 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:20.624903 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025e11f-4fa0-4291-a5c1-fadd4abbca40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:20.635069 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025e11f-4fa0-4291-a5c1-fadd4abbca40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:20.636746 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025e11f-4fa0-4291-a5c1-fadd4abbca40 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:20.644343 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:20.650620 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1005/2040] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:35:20.887015 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec2ec1-24f3-46d7-bf84-b8f3cb98d304 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:20.961209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec2ec1-24f3-46d7-bf84-b8f3cb98d304 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:20.967061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d9eb7-3f2c-45fe-9f1a-d45fef5f4c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:20.967722 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d9eb7-3f2c-45fe-9f1a-d45fef5f4c8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:20.968158 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d9eb7-3f2c-45fe-9f1a-d45fef5f4c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:20.968513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d9eb7-3f2c-45fe-9f1a-d45fef5f4c8d admin admin] PATH_INFO: `/v3/users/a1fe60c8967544c0b849ec9a89c9afbb` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:20.969909 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d9eb7-3f2c-45fe-9f1a-d45fef5f4c8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1fe60c8967544c0b849ec9a89c9afbb)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:20.971913 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d9eb7-3f2c-45fe-9f1a-d45fef5f4c8d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:21.036585 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9b6d9eb7-3f2c-45fe-9f1a-d45fef5f4c8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1fe60c8967544c0b849ec9a89c9afbb'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:21.039103 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9b6d9eb7-3f2c-45fe-9f1a-d45fef5f4c8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1fe60c8967544c0b849ec9a89c9afbb'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:21.057464 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9b6d9eb7-3f2c-45fe-9f1a-d45fef5f4c8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1fe60c8967544c0b849ec9a89c9afbb'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:21.077718 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9b6d9eb7-3f2c-45fe-9f1a-d45fef5f4c8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1fe60c8967544c0b849ec9a89c9afbb'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:21.087070 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9b6d9eb7-3f2c-45fe-9f1a-d45fef5f4c8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1fe60c8967544c0b849ec9a89c9afbb'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:21.093492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447faf27-2547-4bfe-8c6f-6751b5aeeb09 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:21.094030 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447faf27-2547-4bfe-8c6f-6751b5aeeb09 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:21.094107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447faf27-2547-4bfe-8c6f-6751b5aeeb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:21.094948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1036/2041] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:35:20 2022] DELETE /identity/v3/users/a1fe60c8967544c0b849ec9a89c9afbb => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:21.099209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d222d8-976e-4f23-8087-e8c79ef777d2 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:21.099562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d222d8-976e-4f23-8087-e8c79ef777d2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:21.099974 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d222d8-976e-4f23-8087-e8c79ef777d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:21.127684 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-36d222d8-976e-4f23-8087-e8c79ef777d2 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:21.128925 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-36d222d8-976e-4f23-8087-e8c79ef777d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:21.132161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-447faf27-2547-4bfe-8c6f-6751b5aeeb09 None None] MFA Rules not processed for user `77cb30fae1184064a8c4f822d4d31096`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:21.134231 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-447faf27-2547-4bfe-8c6f-6751b5aeeb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:21.160973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:21.206067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1037/2042] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:35:21.216615 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d222d8-976e-4f23-8087-e8c79ef777d2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:21.218215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-36d222d8-976e-4f23-8087-e8c79ef777d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:21.231734 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1006/2043] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:21.242440 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447faf27-2547-4bfe-8c6f-6751b5aeeb09 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:21.243918 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-447faf27-2547-4bfe-8c6f-6751b5aeeb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:21.294547 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d222d8-976e-4f23-8087-e8c79ef777d2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:21.299335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddd442-2db7-445f-8dc5-d2c185d6e4ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:21.300295 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddd442-2db7-445f-8dc5-d2c185d6e4ad service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:21.300712 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddd442-2db7-445f-8dc5-d2c185d6e4ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:21.300991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddd442-2db7-445f-8dc5-d2c185d6e4ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:21.311734 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddd442-2db7-445f-8dc5-d2c185d6e4ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:21.313208 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddd442-2db7-445f-8dc5-d2c185d6e4ad service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:21.316032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447faf27-2547-4bfe-8c6f-6751b5aeeb09 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:21.321516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe412187-0dfb-41e1-bb55-35232d32d80a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:21.322367 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe412187-0dfb-41e1-bb55-35232d32d80a service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:21.322722 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe412187-0dfb-41e1-bb55-35232d32d80a service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:21.323067 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe412187-0dfb-41e1-bb55-35232d32d80a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:21.330052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1038/2044] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:35:21.333657 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe412187-0dfb-41e1-bb55-35232d32d80a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:21.335511 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe412187-0dfb-41e1-bb55-35232d32d80a service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:21.346405 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1007/2045] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:35:21.656730 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddd442-2db7-445f-8dc5-d2c185d6e4ad service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:21.658880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9dddd442-2db7-445f-8dc5-d2c185d6e4ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:21.706207 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddd442-2db7-445f-8dc5-d2c185d6e4ad service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:21.723690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ab1f3-7a14-4ab1-bec8-f8a355959b4c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:21.724243 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ab1f3-7a14-4ab1-bec8-f8a355959b4c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:21.724525 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ab1f3-7a14-4ab1-bec8-f8a355959b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:21.732256 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14972605-41f6-434c-bf36-329bd1fe4004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:21.733344 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14972605-41f6-434c-bf36-329bd1fe4004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:21.733976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14972605-41f6-434c-bf36-329bd1fe4004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:21.734711 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14972605-41f6-434c-bf36-329bd1fe4004 admin admin] PATH_INFO: `/v3/projects/53f7598bdaaf4898acfa6988952d1b91` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:21.736294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14972605-41f6-434c-bf36-329bd1fe4004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53f7598bdaaf4898acfa6988952d1b91)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:21.738898 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14972605-41f6-434c-bf36-329bd1fe4004 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:21.758360 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d80ab1f3-7a14-4ab1-bec8-f8a355959b4c None None] MFA Rules not processed for user `c6591fd1ff63406c86b847870570e56c`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:21.759627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d80ab1f3-7a14-4ab1-bec8-f8a355959b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:21.796504 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-14972605-41f6-434c-bf36-329bd1fe4004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53f7598bdaaf4898acfa6988952d1b91', 'request_id': 'req-14972605-41f6-434c-bf36-329bd1fe4004'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:21.804422 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-14972605-41f6-434c-bf36-329bd1fe4004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53f7598bdaaf4898acfa6988952d1b91', 'request_id': 'req-14972605-41f6-434c-bf36-329bd1fe4004'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:21.814043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-14972605-41f6-434c-bf36-329bd1fe4004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53f7598bdaaf4898acfa6988952d1b91', 'request_id': 'req-14972605-41f6-434c-bf36-329bd1fe4004'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:21.830691 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-14972605-41f6-434c-bf36-329bd1fe4004 admin admin] The token cache is being invalidate because project 53f7598bdaaf4898acfa6988952d1b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:35:21.830983 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-14972605-41f6-434c-bf36-329bd1fe4004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:21.831620 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-14972605-41f6-434c-bf36-329bd1fe4004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:21.833373 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1039/2046] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:35:21 2022] DELETE /identity/v3/projects/53f7598bdaaf4898acfa6988952d1b91 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:21.834594 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1008/2047] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:21.840101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4731af-43d9-4b7b-b1bf-dc2a3a9b2d7c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:21.840434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4731af-43d9-4b7b-b1bf-dc2a3a9b2d7c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:21.840835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4731af-43d9-4b7b-b1bf-dc2a3a9b2d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:21.862950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8b4731af-43d9-4b7b-b1bf-dc2a3a9b2d7c None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:21.864415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8b4731af-43d9-4b7b-b1bf-dc2a3a9b2d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:21.920036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:21.938708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1040/2048] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:21.947595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccf000-83e2-4313-b9af-db988b702c66 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:21.948008 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccf000-83e2-4313-b9af-db988b702c66 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:21.948176 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccf000-83e2-4313-b9af-db988b702c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:21.975525 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-23ccf000-83e2-4313-b9af-db988b702c66 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:21.976793 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-23ccf000-83e2-4313-b9af-db988b702c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:21.985554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1009/2049] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:21.990003 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4731af-43d9-4b7b-b1bf-dc2a3a9b2d7c None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:22.001078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4731af-43d9-4b7b-b1bf-dc2a3a9b2d7c None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:22.005916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6386ff2-7ba4-4a81-9a7a-4f6fff08ca44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:22.006564 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6386ff2-7ba4-4a81-9a7a-4f6fff08ca44 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:22.006846 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6386ff2-7ba4-4a81-9a7a-4f6fff08ca44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:22.007124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6386ff2-7ba4-4a81-9a7a-4f6fff08ca44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:22.007821 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6386ff2-7ba4-4a81-9a7a-4f6fff08ca44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:35:22.008127 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6386ff2-7ba4-4a81-9a7a-4f6fff08ca44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:22.009748 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6386ff2-7ba4-4a81-9a7a-4f6fff08ca44 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:22.022785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1041/2050] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:35:22.027207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccf000-83e2-4313-b9af-db988b702c66 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:22.037297 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccf000-83e2-4313-b9af-db988b702c66 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:22.042318 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4fd14c-cde7-47cc-8f0f-29081ec10c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:22.043004 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fd14c-cde7-47cc-8f0f-29081ec10c1e admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:22.043301 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fd14c-cde7-47cc-8f0f-29081ec10c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:22.043603 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fd14c-cde7-47cc-8f0f-29081ec10c1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:22.044881 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4fd14c-cde7-47cc-8f0f-29081ec10c1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:22.047079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4fd14c-cde7-47cc-8f0f-29081ec10c1e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:22.086013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1010/2051] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:35:22 2022] POST /identity/v3/projects => generated 355 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:22.090211 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6386ff2-7ba4-4a81-9a7a-4f6fff08ca44 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:22.100403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6386ff2-7ba4-4a81-9a7a-4f6fff08ca44 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:22.161799 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805542e1-dcbb-44e2-9c99-dac99892a3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:22.162256 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805542e1-dcbb-44e2-9c99-dac99892a3b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:22.162652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805542e1-dcbb-44e2-9c99-dac99892a3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:22.163245 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805542e1-dcbb-44e2-9c99-dac99892a3b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:22.164376 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805542e1-dcbb-44e2-9c99-dac99892a3b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:22.166616 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805542e1-dcbb-44e2-9c99-dac99892a3b8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:22.200498 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1042/2052] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:35:22 2022] POST /identity/v3/users => generated 278 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:22.205309 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4fd14c-cde7-47cc-8f0f-29081ec10c1e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:22.220676 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4fd14c-cde7-47cc-8f0f-29081ec10c1e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:22.227442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7093b4-81fb-4133-a9ef-200c66e98e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:22.228375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7093b4-81fb-4133-a9ef-200c66e98e72 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:22.228793 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7093b4-81fb-4133-a9ef-200c66e98e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:22.229425 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7093b4-81fb-4133-a9ef-200c66e98e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:22.230542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7093b4-81fb-4133-a9ef-200c66e98e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:22.233619 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7093b4-81fb-4133-a9ef-200c66e98e72 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:22.236980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:22.250137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1011/2053] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:22.254209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805542e1-dcbb-44e2-9c99-dac99892a3b8 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:22.264837 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805542e1-dcbb-44e2-9c99-dac99892a3b8 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:22.270261 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da602cff-b1f1-4908-8108-943a17e1d1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:22.270922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da602cff-b1f1-4908-8108-943a17e1d1b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:22.271205 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da602cff-b1f1-4908-8108-943a17e1d1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:22.271474 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da602cff-b1f1-4908-8108-943a17e1d1b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:22.272201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da602cff-b1f1-4908-8108-943a17e1d1b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:22.273858 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da602cff-b1f1-4908-8108-943a17e1d1b1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:22.285005 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1043/2054] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:35:22.289403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7093b4-81fb-4133-a9ef-200c66e98e72 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:22.300154 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7093b4-81fb-4133-a9ef-200c66e98e72 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:22.305970 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544bf39-8e57-461c-bbdf-71ef01ef6cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:22.306548 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544bf39-8e57-461c-bbdf-71ef01ef6cac admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:22.306836 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544bf39-8e57-461c-bbdf-71ef01ef6cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:22.307118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544bf39-8e57-461c-bbdf-71ef01ef6cac admin admin] PATH_INFO: `/v3/projects/8a2ec0f55ce447a98b82f6b01797c9ba/users/29fe4ae6cea04f8bbe067ace0f5ad04f/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:22.322806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544bf39-8e57-461c-bbdf-71ef01ef6cac admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a2ec0f55ce447a98b82f6b01797c9ba, user_id=29fe4ae6cea04f8bbe067ace0f5ad04f, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:22.325497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544bf39-8e57-461c-bbdf-71ef01ef6cac admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:22.346412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1012/2055] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:35:22 2022] PUT /identity/v3/projects/8a2ec0f55ce447a98b82f6b01797c9ba/users/29fe4ae6cea04f8bbe067ace0f5ad04f/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:35:22.352523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c6045-f87d-4c70-a21f-b5c1ff5bfaa4 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:22.352523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c6045-f87d-4c70-a21f-b5c1ff5bfaa4 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:22.353024 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c6045-f87d-4c70-a21f-b5c1ff5bfaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:22.390034 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-756c6045-f87d-4c70-a21f-b5c1ff5bfaa4 None None] MFA Rules not processed for user `29fe4ae6cea04f8bbe067ace0f5ad04f`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:22.391458 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-756c6045-f87d-4c70-a21f-b5c1ff5bfaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:22.462177 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1044/2056] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:22 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:22.467441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544bf39-8e57-461c-bbdf-71ef01ef6cac admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:22.571777 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544bf39-8e57-461c-bbdf-71ef01ef6cac admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:22.578696 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa56a09-d358-484b-9694-8be1079ef2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:22.579609 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa56a09-d358-484b-9694-8be1079ef2ec admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:22.580152 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa56a09-d358-484b-9694-8be1079ef2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:22.580585 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa56a09-d358-484b-9694-8be1079ef2ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:22.581565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa56a09-d358-484b-9694-8be1079ef2ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:22.584137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa56a09-d358-484b-9694-8be1079ef2ec admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:22.619108 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1013/2057] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:35:22 2022] POST /identity/v3/projects => generated 355 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:22.623079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c6045-f87d-4c70-a21f-b5c1ff5bfaa4 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:22.633985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c6045-f87d-4c70-a21f-b5c1ff5bfaa4 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:22.687887 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:22.699998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d56f7c-959d-4168-bca4-bdf651f53ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:22.701983 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d56f7c-959d-4168-bca4-bdf651f53ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:22.702402 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d56f7c-959d-4168-bca4-bdf651f53ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:22.702679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d56f7c-959d-4168-bca4-bdf651f53ca5 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:22.703372 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d56f7c-959d-4168-bca4-bdf651f53ca5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:22.705318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d56f7c-959d-4168-bca4-bdf651f53ca5 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:22.729932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1045/2058] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:35:22 2022] POST /identity/v3/users => generated 284 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:22.733322 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa56a09-d358-484b-9694-8be1079ef2ec admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:22.747783 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa56a09-d358-484b-9694-8be1079ef2ec admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:22.755004 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfbc58-27a6-4548-b7df-03bffe15455e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:22.755965 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfbc58-27a6-4548-b7df-03bffe15455e admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:22.756323 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfbc58-27a6-4548-b7df-03bffe15455e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:22.756685 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfbc58-27a6-4548-b7df-03bffe15455e admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:22.757472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bfbc58-27a6-4548-b7df-03bffe15455e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:22.759386 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bfbc58-27a6-4548-b7df-03bffe15455e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:22.776978 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1014/2059] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:22.781774 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d56f7c-959d-4168-bca4-bdf651f53ca5 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:22.792549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d56f7c-959d-4168-bca4-bdf651f53ca5 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:22.797781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff927d-7cef-4c8b-80bb-953e3ecef63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:22.798492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff927d-7cef-4c8b-80bb-953e3ecef63d admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:22.798782 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff927d-7cef-4c8b-80bb-953e3ecef63d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:22.799166 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff927d-7cef-4c8b-80bb-953e3ecef63d admin admin] PATH_INFO: `/v3/projects/d90e04c7bb474d65885f28243393c19d/users/b9ad475283f34828bd08d95e001c0c8c/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:22.802021 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c393e-6fc7-4360-81f2-681570cc9a51 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:22.803208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c393e-6fc7-4360-81f2-681570cc9a51 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:22.803563 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c393e-6fc7-4360-81f2-681570cc9a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:22.816959 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff927d-7cef-4c8b-80bb-953e3ecef63d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d90e04c7bb474d65885f28243393c19d, user_id=b9ad475283f34828bd08d95e001c0c8c, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:22.819876 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff927d-7cef-4c8b-80bb-953e3ecef63d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:22.827560 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-0f8c393e-6fc7-4360-81f2-681570cc9a51 None None] MFA Rules not processed for user `77cb30fae1184064a8c4f822d4d31096`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:22.829084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0f8c393e-6fc7-4360-81f2-681570cc9a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:22.835645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1046/2060] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:35:22 2022] PUT /identity/v3/projects/d90e04c7bb474d65885f28243393c19d/users/b9ad475283f34828bd08d95e001c0c8c/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:22.839515 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff927d-7cef-4c8b-80bb-953e3ecef63d admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:22.900200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1015/2061] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:35:22.907785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff927d-7cef-4c8b-80bb-953e3ecef63d admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:22.912586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f301e9-b0e1-44e0-972a-612c68b3d792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:22.913208 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f301e9-b0e1-44e0-972a-612c68b3d792 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:22.913490 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f301e9-b0e1-44e0-972a-612c68b3d792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:22.913768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f301e9-b0e1-44e0-972a-612c68b3d792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:22.914524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f301e9-b0e1-44e0-972a-612c68b3d792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:22.916147 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f301e9-b0e1-44e0-972a-612c68b3d792 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:22.927252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1047/2062] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:35:22.931329 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c393e-6fc7-4360-81f2-681570cc9a51 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:22.942620 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c393e-6fc7-4360-81f2-681570cc9a51 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:22.947543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a64d5-27c0-4a7b-b5e8-64a205c4e2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:22.948314 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a64d5-27c0-4a7b-b5e8-64a205c4e2c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:22.948702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a64d5-27c0-4a7b-b5e8-64a205c4e2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:22.949182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a64d5-27c0-4a7b-b5e8-64a205c4e2c1 admin admin] PATH_INFO: `/v3/projects/d90e04c7bb474d65885f28243393c19d/users/b9ad475283f34828bd08d95e001c0c8c/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:22.952621 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517a64d5-27c0-4a7b-b5e8-64a205c4e2c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d90e04c7bb474d65885f28243393c19d, user_id=b9ad475283f34828bd08d95e001c0c8c, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:22.957195 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517a64d5-27c0-4a7b-b5e8-64a205c4e2c1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:22.973936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1016/2063] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:35:22 2022] PUT /identity/v3/projects/d90e04c7bb474d65885f28243393c19d/users/b9ad475283f34828bd08d95e001c0c8c/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:22.982980 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1f6ed-75b3-425c-95a7-5717643c1e63 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:22.983658 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1f6ed-75b3-425c-95a7-5717643c1e63 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:22.984241 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1f6ed-75b3-425c-95a7-5717643c1e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:23.018803 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-aee1f6ed-75b3-425c-95a7-5717643c1e63 None None] MFA Rules not processed for user `b9ad475283f34828bd08d95e001c0c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:23.020099 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-aee1f6ed-75b3-425c-95a7-5717643c1e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:23.144675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1048/2064] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:22 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:35:23.158268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a64d5-27c0-4a7b-b5e8-64a205c4e2c1 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:23.160308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-517a64d5-27c0-4a7b-b5e8-64a205c4e2c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:23.217281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:23.242022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a64d5-27c0-4a7b-b5e8-64a205c4e2c1 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:23.246076 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62338a28-cc35-4f80-b0e4-77ba85e86154 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:23.246890 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62338a28-cc35-4f80-b0e4-77ba85e86154 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:23.247934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62338a28-cc35-4f80-b0e4-77ba85e86154 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:23.248372 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62338a28-cc35-4f80-b0e4-77ba85e86154 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:23.260293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62338a28-cc35-4f80-b0e4-77ba85e86154 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:23.262248 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62338a28-cc35-4f80-b0e4-77ba85e86154 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:23.274623 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1017/2065] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:23 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:35:23.376182 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1f6ed-75b3-425c-95a7-5717643c1e63 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:23.378238 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-aee1f6ed-75b3-425c-95a7-5717643c1e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:23.401170 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1f6ed-75b3-425c-95a7-5717643c1e63 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:23.405264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d698efe-dd51-4630-9e52-462b2d098401 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:23.405916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d698efe-dd51-4630-9e52-462b2d098401 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:23.406552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d698efe-dd51-4630-9e52-462b2d098401 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:23.406552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d698efe-dd51-4630-9e52-462b2d098401 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:23.454764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:23.476916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d698efe-dd51-4630-9e52-462b2d098401 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:23.478769 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d698efe-dd51-4630-9e52-462b2d098401 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:23.488322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1049/2066] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:23 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:35:33.268915 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=8 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:33.484233 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=10 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:40.493035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62338a28-cc35-4f80-b0e4-77ba85e86154 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:40.494635 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-62338a28-cc35-4f80-b0e4-77ba85e86154 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:40.593680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62338a28-cc35-4f80-b0e4-77ba85e86154 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:40.609300 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1263a-dc49-4127-875a-094223ecafc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:40.610553 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1263a-dc49-4127-875a-094223ecafc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:40.611060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1263a-dc49-4127-875a-094223ecafc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:40.611507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1263a-dc49-4127-875a-094223ecafc5 admin admin] PATH_INFO: `/v3/users/aee1d58902994c5683b1936fc6df756c` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:40.613159 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1263a-dc49-4127-875a-094223ecafc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aee1d58902994c5683b1936fc6df756c)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:40.615428 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1263a-dc49-4127-875a-094223ecafc5 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:40.711349 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-27d1263a-dc49-4127-875a-094223ecafc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aee1d58902994c5683b1936fc6df756c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:40.712400 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-27d1263a-dc49-4127-875a-094223ecafc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aee1d58902994c5683b1936fc6df756c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:40.724125 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-27d1263a-dc49-4127-875a-094223ecafc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aee1d58902994c5683b1936fc6df756c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:40.760141 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-27d1263a-dc49-4127-875a-094223ecafc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aee1d58902994c5683b1936fc6df756c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:40.761796 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-27d1263a-dc49-4127-875a-094223ecafc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aee1d58902994c5683b1936fc6df756c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:40.787601 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1018/2067] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:35:40 2022] DELETE /identity/v3/users/aee1d58902994c5683b1936fc6df756c => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:40.792357 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cbad1-6498-4d63-a907-3886329fb8bd None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:40.792889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cbad1-6498-4d63-a907-3886329fb8bd None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:40.793010 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cbad1-6498-4d63-a907-3886329fb8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:40.815814 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-fb8cbad1-6498-4d63-a907-3886329fb8bd None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:40.817328 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fb8cbad1-6498-4d63-a907-3886329fb8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:40.922157 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1050/2068] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:40.931279 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1263a-dc49-4127-875a-094223ecafc5 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:40.932816 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-27d1263a-dc49-4127-875a-094223ecafc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:40.997375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1263a-dc49-4127-875a-094223ecafc5 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:41.001199 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24571532-6859-49d0-9fb7-4338ea2dcab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:41.001919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24571532-6859-49d0-9fb7-4338ea2dcab8 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:41.002139 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24571532-6859-49d0-9fb7-4338ea2dcab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:41.002412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24571532-6859-49d0-9fb7-4338ea2dcab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:41.014780 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24571532-6859-49d0-9fb7-4338ea2dcab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:41.016529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24571532-6859-49d0-9fb7-4338ea2dcab8 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:41.029152 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1019/2069] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:35:41.328339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8cbad1-6498-4d63-a907-3886329fb8bd None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:41.330181 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fb8cbad1-6498-4d63-a907-3886329fb8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:41.346037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8cbad1-6498-4d63-a907-3886329fb8bd None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:41.351227 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1b91b-e9bf-455c-912f-0226b5515b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:41.351881 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1b91b-e9bf-455c-912f-0226b5515b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:41.352162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1b91b-e9bf-455c-912f-0226b5515b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:41.352434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1b91b-e9bf-455c-912f-0226b5515b48 admin admin] PATH_INFO: `/v3/projects/8396d8e73bc64e688edf7cb10bea233b` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:41.354401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1b91b-e9bf-455c-912f-0226b5515b48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8396d8e73bc64e688edf7cb10bea233b)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:41.356487 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1b91b-e9bf-455c-912f-0226b5515b48 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:41.410590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1cd1b91b-e9bf-455c-912f-0226b5515b48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8396d8e73bc64e688edf7cb10bea233b', 'request_id': 'req-1cd1b91b-e9bf-455c-912f-0226b5515b48'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:41.418160 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1cd1b91b-e9bf-455c-912f-0226b5515b48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8396d8e73bc64e688edf7cb10bea233b', 'request_id': 'req-1cd1b91b-e9bf-455c-912f-0226b5515b48'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:41.425364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1cd1b91b-e9bf-455c-912f-0226b5515b48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8396d8e73bc64e688edf7cb10bea233b', 'request_id': 'req-1cd1b91b-e9bf-455c-912f-0226b5515b48'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:41.442214 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1cd1b91b-e9bf-455c-912f-0226b5515b48 admin admin] The token cache is being invalidate because project 8396d8e73bc64e688edf7cb10bea233b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:35:41.442501 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1cd1b91b-e9bf-455c-912f-0226b5515b48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:41.443158 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1cd1b91b-e9bf-455c-912f-0226b5515b48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:41.444799 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1051/2070] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:35:41 2022] DELETE /identity/v3/projects/8396d8e73bc64e688edf7cb10bea233b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:35:41.450763 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ef86b-2716-4357-a61e-d34dea961c0b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:41.451053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ef86b-2716-4357-a61e-d34dea961c0b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:41.451360 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ef86b-2716-4357-a61e-d34dea961c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:41.472409 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-843ef86b-2716-4357-a61e-d34dea961c0b None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:41.473663 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-843ef86b-2716-4357-a61e-d34dea961c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:41.493484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:41.547495 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1020/2071] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:41.555316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4901ea-4c0e-42b3-88ce-21c42b993e49 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:41.555604 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4901ea-4c0e-42b3-88ce-21c42b993e49 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:41.555884 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4901ea-4c0e-42b3-88ce-21c42b993e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:41.578131 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-da4901ea-4c0e-42b3-88ce-21c42b993e49 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:41.579432 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-da4901ea-4c0e-42b3-88ce-21c42b993e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:41.588538 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1052/2072] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:41.594354 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ef86b-2716-4357-a61e-d34dea961c0b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:41.605570 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ef86b-2716-4357-a61e-d34dea961c0b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:41.612529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a786a58-03d0-405e-8220-f542a32b2c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:41.613264 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a786a58-03d0-405e-8220-f542a32b2c07 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:41.613686 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a786a58-03d0-405e-8220-f542a32b2c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:41.613990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a786a58-03d0-405e-8220-f542a32b2c07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:41.614701 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a786a58-03d0-405e-8220-f542a32b2c07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:35:41.615031 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a786a58-03d0-405e-8220-f542a32b2c07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:41.616712 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a786a58-03d0-405e-8220-f542a32b2c07 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:41.631407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1021/2073] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:35:41.636210 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4901ea-4c0e-42b3-88ce-21c42b993e49 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:41.646234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4901ea-4c0e-42b3-88ce-21c42b993e49 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:41.651851 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8002c7-bc53-475e-bd6d-34bfae49192f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:41.652473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8002c7-bc53-475e-bd6d-34bfae49192f admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:41.653019 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8002c7-bc53-475e-bd6d-34bfae49192f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:41.653019 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8002c7-bc53-475e-bd6d-34bfae49192f admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:41.653692 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8002c7-bc53-475e-bd6d-34bfae49192f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:41.655632 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8002c7-bc53-475e-bd6d-34bfae49192f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:41.702054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1053/2074] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:35:41 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:41.703483 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a786a58-03d0-405e-8220-f542a32b2c07 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:41.719484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a786a58-03d0-405e-8220-f542a32b2c07 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:41.790773 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b653f5-c1a5-4cfe-bb39-a5e0da89c49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:41.791421 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b653f5-c1a5-4cfe-bb39-a5e0da89c49a admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:41.791694 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b653f5-c1a5-4cfe-bb39-a5e0da89c49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:41.791967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b653f5-c1a5-4cfe-bb39-a5e0da89c49a admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:41.792632 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b653f5-c1a5-4cfe-bb39-a5e0da89c49a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:41.795147 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b653f5-c1a5-4cfe-bb39-a5e0da89c49a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:41.817967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1022/2075] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:35:41 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:41.822323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8002c7-bc53-475e-bd6d-34bfae49192f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:41.833135 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8002c7-bc53-475e-bd6d-34bfae49192f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:41.838075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36562502-f6fb-40de-ba80-1a77a60a5123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:41.838722 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36562502-f6fb-40de-ba80-1a77a60a5123 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:41.839000 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36562502-f6fb-40de-ba80-1a77a60a5123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:41.839317 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36562502-f6fb-40de-ba80-1a77a60a5123 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:41.839993 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36562502-f6fb-40de-ba80-1a77a60a5123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:41.842125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36562502-f6fb-40de-ba80-1a77a60a5123 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:41.852658 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1054/2076] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:41.857523 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b653f5-c1a5-4cfe-bb39-a5e0da89c49a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:41.869090 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b653f5-c1a5-4cfe-bb39-a5e0da89c49a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:41.874046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ffc37-279c-4387-8716-2eaf1d78c592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:41.874692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ffc37-279c-4387-8716-2eaf1d78c592 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:41.874955 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ffc37-279c-4387-8716-2eaf1d78c592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:41.875246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ffc37-279c-4387-8716-2eaf1d78c592 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:41.875948 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ffc37-279c-4387-8716-2eaf1d78c592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:41.877577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ffc37-279c-4387-8716-2eaf1d78c592 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:41.888992 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1023/2077] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:41.893604 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36562502-f6fb-40de-ba80-1a77a60a5123 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:41.905495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36562502-f6fb-40de-ba80-1a77a60a5123 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:41.910367 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f66d1-5dc0-4d77-adf2-5ebd8142dfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:41.911275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f66d1-5dc0-4d77-adf2-5ebd8142dfe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:41.911563 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f66d1-5dc0-4d77-adf2-5ebd8142dfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:41.911699 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f66d1-5dc0-4d77-adf2-5ebd8142dfe6 admin admin] PATH_INFO: `/v3/projects/486d4d85b8b34a2f926210b6d088b33b/users/94c1d50cb45f4747b1e4388290dfc0b3/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:41.929338 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f66d1-5dc0-4d77-adf2-5ebd8142dfe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=486d4d85b8b34a2f926210b6d088b33b, user_id=94c1d50cb45f4747b1e4388290dfc0b3, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:41.932047 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f66d1-5dc0-4d77-adf2-5ebd8142dfe6 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:41.948052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1055/2078] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:35:41 2022] PUT /identity/v3/projects/486d4d85b8b34a2f926210b6d088b33b/users/94c1d50cb45f4747b1e4388290dfc0b3/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:41.958728 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d546d-ff38-4680-9b40-ef40849f58a8 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:41.959683 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d546d-ff38-4680-9b40-ef40849f58a8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:41.959683 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d546d-ff38-4680-9b40-ef40849f58a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:41.991413 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-b12d546d-ff38-4680-9b40-ef40849f58a8 None None] MFA Rules not processed for user `94c1d50cb45f4747b1e4388290dfc0b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:41.992706 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b12d546d-ff38-4680-9b40-ef40849f58a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:42.101767 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1024/2079] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:42.114492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f66d1-5dc0-4d77-adf2-5ebd8142dfe6 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:42.116446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-807f66d1-5dc0-4d77-adf2-5ebd8142dfe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:42.154740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:42.197389 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f66d1-5dc0-4d77-adf2-5ebd8142dfe6 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:42.203895 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff382ff4-93a6-4055-b39d-ce1d492348dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:42.204701 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff382ff4-93a6-4055-b39d-ce1d492348dc service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:42.204983 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff382ff4-93a6-4055-b39d-ce1d492348dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:42.205254 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff382ff4-93a6-4055-b39d-ce1d492348dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:42.218966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff382ff4-93a6-4055-b39d-ce1d492348dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:42.220498 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff382ff4-93a6-4055-b39d-ce1d492348dc service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:42.231398 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1056/2080] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:35:42.477617 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d546d-ff38-4680-9b40-ef40849f58a8 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:42.479765 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b12d546d-ff38-4680-9b40-ef40849f58a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:42.502393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:42.553799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d546d-ff38-4680-9b40-ef40849f58a8 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:42.558583 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ad981-1090-465d-bed9-c78cef3d65f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:42.559290 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ad981-1090-465d-bed9-c78cef3d65f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:42.559577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ad981-1090-465d-bed9-c78cef3d65f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:42.559853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ad981-1090-465d-bed9-c78cef3d65f1 admin admin] PATH_INFO: `/v3/users/29fe4ae6cea04f8bbe067ace0f5ad04f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:42.561020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ad981-1090-465d-bed9-c78cef3d65f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29fe4ae6cea04f8bbe067ace0f5ad04f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:42.562987 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ad981-1090-465d-bed9-c78cef3d65f1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:42.619664 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3c3ad981-1090-465d-bed9-c78cef3d65f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29fe4ae6cea04f8bbe067ace0f5ad04f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:42.620404 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3c3ad981-1090-465d-bed9-c78cef3d65f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29fe4ae6cea04f8bbe067ace0f5ad04f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:42.631158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3c3ad981-1090-465d-bed9-c78cef3d65f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29fe4ae6cea04f8bbe067ace0f5ad04f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:42.660644 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3c3ad981-1090-465d-bed9-c78cef3d65f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29fe4ae6cea04f8bbe067ace0f5ad04f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:42.661630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3c3ad981-1090-465d-bed9-c78cef3d65f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29fe4ae6cea04f8bbe067ace0f5ad04f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:42.678164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1025/2081] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:35:42 2022] DELETE /identity/v3/users/29fe4ae6cea04f8bbe067ace0f5ad04f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:42.682987 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38651d-5c98-48c6-9eb6-bb7a16f1ebfd None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:42.683302 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38651d-5c98-48c6-9eb6-bb7a16f1ebfd None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:42.683584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38651d-5c98-48c6-9eb6-bb7a16f1ebfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:42.709384 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-9b38651d-5c98-48c6-9eb6-bb7a16f1ebfd None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:42.710899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9b38651d-5c98-48c6-9eb6-bb7a16f1ebfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:42.776568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1057/2082] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:42.787166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ad981-1090-465d-bed9-c78cef3d65f1 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:42.788725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3c3ad981-1090-465d-bed9-c78cef3d65f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:42.860596 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ad981-1090-465d-bed9-c78cef3d65f1 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:42.866186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1c0c1-9e3f-4c20-8c29-d53123c0206a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:42.866186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1c0c1-9e3f-4c20-8c29-d53123c0206a service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:42.866186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1c0c1-9e3f-4c20-8c29-d53123c0206a service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:42.866186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1c0c1-9e3f-4c20-8c29-d53123c0206a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:42.877223 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1c0c1-9e3f-4c20-8c29-d53123c0206a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:42.878641 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1c0c1-9e3f-4c20-8c29-d53123c0206a service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:42.890033 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1026/2083] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:35:43.405445 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38651d-5c98-48c6-9eb6-bb7a16f1ebfd None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:43.406910 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9b38651d-5c98-48c6-9eb6-bb7a16f1ebfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:43.419907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38651d-5c98-48c6-9eb6-bb7a16f1ebfd None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:43.424848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd54618-dbe7-4107-89c4-9c8f2f1d8fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:43.425513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd54618-dbe7-4107-89c4-9c8f2f1d8fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:43.425793 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd54618-dbe7-4107-89c4-9c8f2f1d8fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:43.426104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd54618-dbe7-4107-89c4-9c8f2f1d8fa2 admin admin] PATH_INFO: `/v3/projects/8a2ec0f55ce447a98b82f6b01797c9ba` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:43.427311 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd54618-dbe7-4107-89c4-9c8f2f1d8fa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a2ec0f55ce447a98b82f6b01797c9ba)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:43.429224 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd54618-dbe7-4107-89c4-9c8f2f1d8fa2 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:43.531439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8cd54618-dbe7-4107-89c4-9c8f2f1d8fa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a2ec0f55ce447a98b82f6b01797c9ba', 'request_id': 'req-8cd54618-dbe7-4107-89c4-9c8f2f1d8fa2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:43.537250 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8cd54618-dbe7-4107-89c4-9c8f2f1d8fa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a2ec0f55ce447a98b82f6b01797c9ba', 'request_id': 'req-8cd54618-dbe7-4107-89c4-9c8f2f1d8fa2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:43.544769 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8cd54618-dbe7-4107-89c4-9c8f2f1d8fa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a2ec0f55ce447a98b82f6b01797c9ba', 'request_id': 'req-8cd54618-dbe7-4107-89c4-9c8f2f1d8fa2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:43.560058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8cd54618-dbe7-4107-89c4-9c8f2f1d8fa2 admin admin] The token cache is being invalidate because project 8a2ec0f55ce447a98b82f6b01797c9ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:35:43.560639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8cd54618-dbe7-4107-89c4-9c8f2f1d8fa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:43.561092 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8cd54618-dbe7-4107-89c4-9c8f2f1d8fa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:43.562667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1058/2084] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:35:43 2022] DELETE /identity/v3/projects/8a2ec0f55ce447a98b82f6b01797c9ba => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:43.566693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1c0c1-9e3f-4c20-8c29-d53123c0206a service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:43.576454 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1c0c1-9e3f-4c20-8c29-d53123c0206a service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:43.617788 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:43.639886 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630c900-458d-469a-a93c-d933fc8ff51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:43.640538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630c900-458d-469a-a93c-d933fc8ff51a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:43.640820 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630c900-458d-469a-a93c-d933fc8ff51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:43.641090 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630c900-458d-469a-a93c-d933fc8ff51a admin admin] PATH_INFO: `/v3/users/b9ad475283f34828bd08d95e001c0c8c` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:43.642975 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2630c900-458d-469a-a93c-d933fc8ff51a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9ad475283f34828bd08d95e001c0c8c)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:43.644979 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2630c900-458d-469a-a93c-d933fc8ff51a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:43.706696 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2630c900-458d-469a-a93c-d933fc8ff51a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9ad475283f34828bd08d95e001c0c8c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:43.707448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2630c900-458d-469a-a93c-d933fc8ff51a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9ad475283f34828bd08d95e001c0c8c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:43.717926 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2630c900-458d-469a-a93c-d933fc8ff51a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9ad475283f34828bd08d95e001c0c8c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:43.731409 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2630c900-458d-469a-a93c-d933fc8ff51a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9ad475283f34828bd08d95e001c0c8c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:43.732141 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2630c900-458d-469a-a93c-d933fc8ff51a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9ad475283f34828bd08d95e001c0c8c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:43.746743 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1027/2085] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:35:43 2022] DELETE /identity/v3/users/b9ad475283f34828bd08d95e001c0c8c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:35:44.384944 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd54618-dbe7-4107-89c4-9c8f2f1d8fa2 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:44.386673 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8cd54618-dbe7-4107-89c4-9c8f2f1d8fa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:44.401111 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd54618-dbe7-4107-89c4-9c8f2f1d8fa2 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:44.452922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:44.471419 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dff727-2b08-47e8-8808-f815d114fdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:44.472080 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dff727-2b08-47e8-8808-f815d114fdd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:44.472358 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dff727-2b08-47e8-8808-f815d114fdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:44.472634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dff727-2b08-47e8-8808-f815d114fdd8 admin admin] PATH_INFO: `/v3/projects/d90e04c7bb474d65885f28243393c19d` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:44.473848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dff727-2b08-47e8-8808-f815d114fdd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d90e04c7bb474d65885f28243393c19d)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:44.475943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dff727-2b08-47e8-8808-f815d114fdd8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:44.537517 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-60dff727-2b08-47e8-8808-f815d114fdd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd90e04c7bb474d65885f28243393c19d', 'request_id': 'req-60dff727-2b08-47e8-8808-f815d114fdd8'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:44.544664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-60dff727-2b08-47e8-8808-f815d114fdd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd90e04c7bb474d65885f28243393c19d', 'request_id': 'req-60dff727-2b08-47e8-8808-f815d114fdd8'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:44.552640 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-60dff727-2b08-47e8-8808-f815d114fdd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd90e04c7bb474d65885f28243393c19d', 'request_id': 'req-60dff727-2b08-47e8-8808-f815d114fdd8'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:44.573380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-60dff727-2b08-47e8-8808-f815d114fdd8 admin admin] The token cache is being invalidate because project d90e04c7bb474d65885f28243393c19d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:35:44.573661 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-60dff727-2b08-47e8-8808-f815d114fdd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:44.575169 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-60dff727-2b08-47e8-8808-f815d114fdd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:44.577217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1059/2086] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:35:44 2022] DELETE /identity/v3/projects/d90e04c7bb474d65885f28243393c19d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:44.583284 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872679b-3e65-414c-9a04-0d0dbb7c18b9 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:44.584074 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872679b-3e65-414c-9a04-0d0dbb7c18b9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:44.584074 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872679b-3e65-414c-9a04-0d0dbb7c18b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:44.610149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3872679b-3e65-414c-9a04-0d0dbb7c18b9 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:44.611433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3872679b-3e65-414c-9a04-0d0dbb7c18b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:44.689683 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1028/2087] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:44.697319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63eb8a-7b03-4c4a-8af2-38b27136eed4 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:44.697606 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63eb8a-7b03-4c4a-8af2-38b27136eed4 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:44.697936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63eb8a-7b03-4c4a-8af2-38b27136eed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:44.721291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-fa63eb8a-7b03-4c4a-8af2-38b27136eed4 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:44.722701 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fa63eb8a-7b03-4c4a-8af2-38b27136eed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:44.733274 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1060/2088] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:44.737526 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872679b-3e65-414c-9a04-0d0dbb7c18b9 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:44.748108 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872679b-3e65-414c-9a04-0d0dbb7c18b9 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:44.752768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca5ad0-873a-404e-bdec-a0c6aa04f26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:44.753387 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca5ad0-873a-404e-bdec-a0c6aa04f26e admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:44.753668 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca5ad0-873a-404e-bdec-a0c6aa04f26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:44.753986 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca5ad0-873a-404e-bdec-a0c6aa04f26e admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:44.754681 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca5ad0-873a-404e-bdec-a0c6aa04f26e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:35:44.754982 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca5ad0-873a-404e-bdec-a0c6aa04f26e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:44.756576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca5ad0-873a-404e-bdec-a0c6aa04f26e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:44.769379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1029/2089] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:35:44.773671 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63eb8a-7b03-4c4a-8af2-38b27136eed4 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:44.783340 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63eb8a-7b03-4c4a-8af2-38b27136eed4 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:44.788607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77de56-8dc4-4854-ae76-053ad46c92bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:44.789323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77de56-8dc4-4854-ae76-053ad46c92bd admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:44.789678 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77de56-8dc4-4854-ae76-053ad46c92bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:44.790067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77de56-8dc4-4854-ae76-053ad46c92bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:44.790875 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77de56-8dc4-4854-ae76-053ad46c92bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:44.792897 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77de56-8dc4-4854-ae76-053ad46c92bd admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:44.825920 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1061/2090] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:35:44 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:44.828259 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca5ad0-873a-404e-bdec-a0c6aa04f26e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:44.839246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca5ad0-873a-404e-bdec-a0c6aa04f26e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:44.893224 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:44.921210 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef459e2-cd8b-410a-b1f1-2780b00590c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:44.922149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef459e2-cd8b-410a-b1f1-2780b00590c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:44.922437 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef459e2-cd8b-410a-b1f1-2780b00590c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:44.922712 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef459e2-cd8b-410a-b1f1-2780b00590c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:44.923478 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef459e2-cd8b-410a-b1f1-2780b00590c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:44.925539 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef459e2-cd8b-410a-b1f1-2780b00590c8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:44.949919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1030/2091] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:35:44 2022] POST /identity/v3/users => generated 294 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:44.954253 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77de56-8dc4-4854-ae76-053ad46c92bd admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:44.966225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77de56-8dc4-4854-ae76-053ad46c92bd admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:44.971051 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e74376-31e5-4ca8-8280-dc5ce0e39d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:44.971610 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e74376-31e5-4ca8-8280-dc5ce0e39d48 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:44.971894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e74376-31e5-4ca8-8280-dc5ce0e39d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:44.972172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e74376-31e5-4ca8-8280-dc5ce0e39d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:44.972876 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e74376-31e5-4ca8-8280-dc5ce0e39d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:44.974528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e74376-31e5-4ca8-8280-dc5ce0e39d48 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:44.985728 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1062/2092] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:44.989962 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef459e2-cd8b-410a-b1f1-2780b00590c8 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:45.005853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef459e2-cd8b-410a-b1f1-2780b00590c8 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:45.015445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93e1e8-d7d5-4472-bcb0-c95615237dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:45.016611 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93e1e8-d7d5-4472-bcb0-c95615237dcb admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:45.017209 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93e1e8-d7d5-4472-bcb0-c95615237dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:45.017706 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93e1e8-d7d5-4472-bcb0-c95615237dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:45.019046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93e1e8-d7d5-4472-bcb0-c95615237dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:45.022456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93e1e8-d7d5-4472-bcb0-c95615237dcb admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:45.040933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1031/2093] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:45.046505 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e74376-31e5-4ca8-8280-dc5ce0e39d48 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:45.058819 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e74376-31e5-4ca8-8280-dc5ce0e39d48 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:45.065718 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490dbf8-cc7e-4a74-aa42-8db45b247b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:45.066518 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490dbf8-cc7e-4a74-aa42-8db45b247b24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:45.066807 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490dbf8-cc7e-4a74-aa42-8db45b247b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:45.067084 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490dbf8-cc7e-4a74-aa42-8db45b247b24 admin admin] PATH_INFO: `/v3/projects/b0824c1b01aa41288e86f341565c5409/users/e6c34715a6c54eaea82189e5150013a9/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:45.084199 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1490dbf8-cc7e-4a74-aa42-8db45b247b24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0824c1b01aa41288e86f341565c5409, user_id=e6c34715a6c54eaea82189e5150013a9, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:45.089180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1490dbf8-cc7e-4a74-aa42-8db45b247b24 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:45.109633 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1063/2094] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:35:45 2022] PUT /identity/v3/projects/b0824c1b01aa41288e86f341565c5409/users/e6c34715a6c54eaea82189e5150013a9/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:35:45.122944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785578b-fe05-40aa-a925-e2b55ef12782 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:45.123260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785578b-fe05-40aa-a925-e2b55ef12782 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:45.123536 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785578b-fe05-40aa-a925-e2b55ef12782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:45.176562 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4785578b-fe05-40aa-a925-e2b55ef12782 None None] MFA Rules not processed for user `e6c34715a6c54eaea82189e5150013a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:45.178132 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4785578b-fe05-40aa-a925-e2b55ef12782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:45.301324 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1032/2095] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:45.308608 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490dbf8-cc7e-4a74-aa42-8db45b247b24 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:45.413089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490dbf8-cc7e-4a74-aa42-8db45b247b24 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:45.419137 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96851736-302b-4cd7-997e-05ad78946cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:45.419789 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96851736-302b-4cd7-997e-05ad78946cfc admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:45.420084 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96851736-302b-4cd7-997e-05ad78946cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:45.420371 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96851736-302b-4cd7-997e-05ad78946cfc admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:45.421060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96851736-302b-4cd7-997e-05ad78946cfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:45.423072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96851736-302b-4cd7-997e-05ad78946cfc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:45.457260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:45.483115 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1064/2096] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:35:45 2022] POST /identity/v3/projects => generated 387 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:45.489859 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785578b-fe05-40aa-a925-e2b55ef12782 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:45.523503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785578b-fe05-40aa-a925-e2b55ef12782 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:45.633204 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf49d94-83d4-4c29-aa9d-b396c5060f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:45.634253 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49d94-83d4-4c29-aa9d-b396c5060f6e admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:45.634581 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49d94-83d4-4c29-aa9d-b396c5060f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:45.634581 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49d94-83d4-4c29-aa9d-b396c5060f6e admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:45.635284 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf49d94-83d4-4c29-aa9d-b396c5060f6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:45.637966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf49d94-83d4-4c29-aa9d-b396c5060f6e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:45.668408 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1033/2097] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:35:45 2022] POST /identity/v3/users => generated 294 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:45.677361 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96851736-302b-4cd7-997e-05ad78946cfc admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:45.697808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96851736-302b-4cd7-997e-05ad78946cfc admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:45.708711 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d0232-86b4-4e8c-bf84-5849a5ab7bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:45.709832 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d0232-86b4-4e8c-bf84-5849a5ab7bcd admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:45.710388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d0232-86b4-4e8c-bf84-5849a5ab7bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:45.711304 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d0232-86b4-4e8c-bf84-5849a5ab7bcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:45.712557 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d0232-86b4-4e8c-bf84-5849a5ab7bcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:45.716573 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d0232-86b4-4e8c-bf84-5849a5ab7bcd admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:45.742043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1065/2098] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:45.744810 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf49d94-83d4-4c29-aa9d-b396c5060f6e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:45.768570 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf49d94-83d4-4c29-aa9d-b396c5060f6e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:45.778470 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ca690-b405-4327-bc83-7c4470b8468c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:45.779564 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ca690-b405-4327-bc83-7c4470b8468c admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:45.780082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ca690-b405-4327-bc83-7c4470b8468c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:45.780882 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ca690-b405-4327-bc83-7c4470b8468c admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:45.781958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ca690-b405-4327-bc83-7c4470b8468c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:45.785235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ca690-b405-4327-bc83-7c4470b8468c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:45.806018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1034/2099] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:35:45.810748 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d0232-86b4-4e8c-bf84-5849a5ab7bcd admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:45.829116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d0232-86b4-4e8c-bf84-5849a5ab7bcd admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:45.837415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc35160-40f6-43b9-851b-5e1eeafe8b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:45.838680 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc35160-40f6-43b9-851b-5e1eeafe8b13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:45.839195 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc35160-40f6-43b9-851b-5e1eeafe8b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:45.839704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc35160-40f6-43b9-851b-5e1eeafe8b13 admin admin] PATH_INFO: `/v3/projects/27b70249fbb641f3b38bd6860f9c73af/users/2ec954936a0b48ca874b064af1ebfee3/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:45.864765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc35160-40f6-43b9-851b-5e1eeafe8b13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b70249fbb641f3b38bd6860f9c73af, user_id=2ec954936a0b48ca874b064af1ebfee3, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:45.870549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc35160-40f6-43b9-851b-5e1eeafe8b13 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:45.889815 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1066/2100] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:35:45 2022] PUT /identity/v3/projects/27b70249fbb641f3b38bd6860f9c73af/users/2ec954936a0b48ca874b064af1ebfee3/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:35:45.901386 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e576a-44e8-4171-a707-11c50190aed2 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:45.901680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e576a-44e8-4171-a707-11c50190aed2 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:45.902049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e576a-44e8-4171-a707-11c50190aed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:45.937139 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-eb1e576a-44e8-4171-a707-11c50190aed2 None None] MFA Rules not processed for user `2ec954936a0b48ca874b064af1ebfee3`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:45.938463 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-eb1e576a-44e8-4171-a707-11c50190aed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:45.960132 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:46.013499 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1035/2101] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:46.018197 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc35160-40f6-43b9-851b-5e1eeafe8b13 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:46.086713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc35160-40f6-43b9-851b-5e1eeafe8b13 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:46.091659 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64c8dc-788a-437e-baf1-7efa354b4390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:46.092278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64c8dc-788a-437e-baf1-7efa354b4390 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:46.092557 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64c8dc-788a-437e-baf1-7efa354b4390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:46.092827 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64c8dc-788a-437e-baf1-7efa354b4390 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:46.093523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64c8dc-788a-437e-baf1-7efa354b4390 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:46.095495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64c8dc-788a-437e-baf1-7efa354b4390 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:46.123904 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1067/2102] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:35:46 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:46.128524 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e576a-44e8-4171-a707-11c50190aed2 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:46.140151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e576a-44e8-4171-a707-11c50190aed2 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:46.201110 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0146cc21-04cd-451e-a16b-6ef2ca969dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:46.201743 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0146cc21-04cd-451e-a16b-6ef2ca969dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:46.202069 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0146cc21-04cd-451e-a16b-6ef2ca969dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:46.202417 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0146cc21-04cd-451e-a16b-6ef2ca969dd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:46.203022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0146cc21-04cd-451e-a16b-6ef2ca969dd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:46.204967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0146cc21-04cd-451e-a16b-6ef2ca969dd8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:46.230792 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1036/2103] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:35:46 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:46.235686 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64c8dc-788a-437e-baf1-7efa354b4390 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:46.246727 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64c8dc-788a-437e-baf1-7efa354b4390 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:46.252224 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1e6a0-59b7-4e91-86ce-27dabc24068d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:46.252860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1e6a0-59b7-4e91-86ce-27dabc24068d admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:46.253140 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1e6a0-59b7-4e91-86ce-27dabc24068d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:46.253415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1e6a0-59b7-4e91-86ce-27dabc24068d admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:46.254153 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1e6a0-59b7-4e91-86ce-27dabc24068d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:46.255798 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1e6a0-59b7-4e91-86ce-27dabc24068d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:46.268513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1068/2104] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:46.272516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0146cc21-04cd-451e-a16b-6ef2ca969dd8 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:46.283019 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0146cc21-04cd-451e-a16b-6ef2ca969dd8 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:46.287804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3fd76-5d83-4d5d-bce6-1e218bedaa40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:46.288488 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3fd76-5d83-4d5d-bce6-1e218bedaa40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:46.288771 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3fd76-5d83-4d5d-bce6-1e218bedaa40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:46.289050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3fd76-5d83-4d5d-bce6-1e218bedaa40 admin admin] PATH_INFO: `/v3/projects/9c1202d7ef6f4ae491f30971e23efe28/users/3e1680179f0b478ca987ed69959c1636/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:46.307109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3fd76-5d83-4d5d-bce6-1e218bedaa40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c1202d7ef6f4ae491f30971e23efe28, user_id=3e1680179f0b478ca987ed69959c1636, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:46.310637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3fd76-5d83-4d5d-bce6-1e218bedaa40 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:46.327393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1037/2105] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:35:46 2022] PUT /identity/v3/projects/9c1202d7ef6f4ae491f30971e23efe28/users/3e1680179f0b478ca987ed69959c1636/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:46.331364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1e6a0-59b7-4e91-86ce-27dabc24068d admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:46.343294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1e6a0-59b7-4e91-86ce-27dabc24068d admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:46.405148 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712128c7-c4a4-4315-a42c-270fafe746c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:46.405788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712128c7-c4a4-4315-a42c-270fafe746c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:46.406098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712128c7-c4a4-4315-a42c-270fafe746c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:46.406368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712128c7-c4a4-4315-a42c-270fafe746c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:46.407054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712128c7-c4a4-4315-a42c-270fafe746c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:46.408664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712128c7-c4a4-4315-a42c-270fafe746c9 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:46.419696 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1069/2106] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:46.423897 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3fd76-5d83-4d5d-bce6-1e218bedaa40 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:46.434454 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3fd76-5d83-4d5d-bce6-1e218bedaa40 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:46.439151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c6a5d-31ad-4daf-b2c6-4ad1583471db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:46.439809 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c6a5d-31ad-4daf-b2c6-4ad1583471db admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:46.440090 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c6a5d-31ad-4daf-b2c6-4ad1583471db admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:46.440363 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c6a5d-31ad-4daf-b2c6-4ad1583471db admin admin] PATH_INFO: `/v3/projects/9c1202d7ef6f4ae491f30971e23efe28/users/3e1680179f0b478ca987ed69959c1636/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:46.442327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c6a5d-31ad-4daf-b2c6-4ad1583471db admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c1202d7ef6f4ae491f30971e23efe28, user_id=3e1680179f0b478ca987ed69959c1636, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:46.445061 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c6a5d-31ad-4daf-b2c6-4ad1583471db admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:46.455504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1038/2107] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:35:46 2022] PUT /identity/v3/projects/9c1202d7ef6f4ae491f30971e23efe28/users/3e1680179f0b478ca987ed69959c1636/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:46.466592 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c21bc-6fd6-4979-8704-08c0773d711e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:46.467119 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c21bc-6fd6-4979-8704-08c0773d711e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:46.467119 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c21bc-6fd6-4979-8704-08c0773d711e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:46.498676 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8e2c21bc-6fd6-4979-8704-08c0773d711e None None] MFA Rules not processed for user `3e1680179f0b478ca987ed69959c1636`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:46.499915 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8e2c21bc-6fd6-4979-8704-08c0773d711e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:46.520367 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:46.573766 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1070/2108] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:46.584689 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c6a5d-31ad-4daf-b2c6-4ad1583471db admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:46.586771 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-523c6a5d-31ad-4daf-b2c6-4ad1583471db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:46.651643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c6a5d-31ad-4daf-b2c6-4ad1583471db admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:46.655854 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8e384-7c63-4535-85be-6f3cc53abb5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:46.656482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8e384-7c63-4535-85be-6f3cc53abb5e service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:46.656761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8e384-7c63-4535-85be-6f3cc53abb5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:46.657031 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8e384-7c63-4535-85be-6f3cc53abb5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:46.667051 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d8e384-7c63-4535-85be-6f3cc53abb5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:46.668534 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d8e384-7c63-4535-85be-6f3cc53abb5e service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:46.682761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1039/2109] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:35:47.463701 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c21bc-6fd6-4979-8704-08c0773d711e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:47.489118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c21bc-6fd6-4979-8704-08c0773d711e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:47.494320 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4f9cf-d6e2-4933-b696-2f4be3b8b760 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:47.495431 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4f9cf-d6e2-4933-b696-2f4be3b8b760 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:47.495982 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4f9cf-d6e2-4933-b696-2f4be3b8b760 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:47.496484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4f9cf-d6e2-4933-b696-2f4be3b8b760 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:47.581868 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4f9cf-d6e2-4933-b696-2f4be3b8b760 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:47.583470 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4f9cf-d6e2-4933-b696-2f4be3b8b760 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:47.594782 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1071/2110] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:35:53.896853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0644c74-5f34-4fe1-979c-999b1f87f4e9 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:53.900456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0644c74-5f34-4fe1-979c-999b1f87f4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:53.900741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0644c74-5f34-4fe1-979c-999b1f87f4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:53.929979 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f0644c74-5f34-4fe1-979c-999b1f87f4e9 None None] MFA Rules not processed for user `f36c6987ef774a1684c55f885e2a4f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:53.932166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f0644c74-5f34-4fe1-979c-999b1f87f4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:53.951557 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:54.001934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1040/2111] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Aug 1 15:35:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:54.057814 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4f9cf-d6e2-4933-b696-2f4be3b8b760 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:54.059839 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-eae4f9cf-d6e2-4933-b696-2f4be3b8b760 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:54.152447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4f9cf-d6e2-4933-b696-2f4be3b8b760 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:54.160291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe50b3a-8cd8-42fb-bcca-d27e49b12632 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6873a3e440bd4b63b0082fc85beef49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:54.162634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe50b3a-8cd8-42fb-bcca-d27e49b12632 service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:54.162934 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe50b3a-8cd8-42fb-bcca-d27e49b12632 service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:54.163241 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe50b3a-8cd8-42fb-bcca-d27e49b12632 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:54.174598 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe50b3a-8cd8-42fb-bcca-d27e49b12632 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:54.176358 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe50b3a-8cd8-42fb-bcca-d27e49b12632 service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:54.185910 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1072/2112] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Aug 1 15:35:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:35:55.150890 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0644c74-5f34-4fe1-979c-999b1f87f4e9 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:55.167916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0644c74-5f34-4fe1-979c-999b1f87f4e9 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:55.173610 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb9223-627e-4063-a94e-03dd0a51bb04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:55.174775 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb9223-627e-4063-a94e-03dd0a51bb04 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:55.175499 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb9223-627e-4063-a94e-03dd0a51bb04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:55.176174 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb9223-627e-4063-a94e-03dd0a51bb04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:55.253171 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb9223-627e-4063-a94e-03dd0a51bb04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:55.255078 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb9223-627e-4063-a94e-03dd0a51bb04 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:55.278708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1041/2113] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:35:56.203991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe50b3a-8cd8-42fb-bcca-d27e49b12632 service swift] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:56.205601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3fe50b3a-8cd8-42fb-bcca-d27e49b12632 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:56.258215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:56.275772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe50b3a-8cd8-42fb-bcca-d27e49b12632 service swift] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:56.281122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19f721-0b14-41dd-b865-23c23bd9c49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:56.281767 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19f721-0b14-41dd-b865-23c23bd9c49b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:56.282083 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19f721-0b14-41dd-b865-23c23bd9c49b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:56.282362 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19f721-0b14-41dd-b865-23c23bd9c49b admin admin] PATH_INFO: `/v3/users/94c1d50cb45f4747b1e4388290dfc0b3` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:56.283587 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19f721-0b14-41dd-b865-23c23bd9c49b admin admin] RBAC: Authorizing `identity:delete_user(user_id=94c1d50cb45f4747b1e4388290dfc0b3)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:56.285523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19f721-0b14-41dd-b865-23c23bd9c49b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:56.347693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1a19f721-0b14-41dd-b865-23c23bd9c49b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94c1d50cb45f4747b1e4388290dfc0b3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:56.348640 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1a19f721-0b14-41dd-b865-23c23bd9c49b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94c1d50cb45f4747b1e4388290dfc0b3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:56.367265 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1a19f721-0b14-41dd-b865-23c23bd9c49b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94c1d50cb45f4747b1e4388290dfc0b3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:56.381626 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1a19f721-0b14-41dd-b865-23c23bd9c49b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94c1d50cb45f4747b1e4388290dfc0b3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:56.389915 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1a19f721-0b14-41dd-b865-23c23bd9c49b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94c1d50cb45f4747b1e4388290dfc0b3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:56.396828 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1073/2114] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:35:56 2022] DELETE /identity/v3/users/94c1d50cb45f4747b1e4388290dfc0b3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:56.402564 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba64df-1df5-44d8-a7e6-88d23c1e5365 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:56.403944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba64df-1df5-44d8-a7e6-88d23c1e5365 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:56.404508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba64df-1df5-44d8-a7e6-88d23c1e5365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:56.428835 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-65ba64df-1df5-44d8-a7e6-88d23c1e5365 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:56.431365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-65ba64df-1df5-44d8-a7e6-88d23c1e5365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:56.516389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1042/2115] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:56.527714 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19f721-0b14-41dd-b865-23c23bd9c49b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:56.530202 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1a19f721-0b14-41dd-b865-23c23bd9c49b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:56.596812 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19f721-0b14-41dd-b865-23c23bd9c49b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:56.600887 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fbdf8-a2b3-431f-9474-d7e5f03c0f07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:56.601511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fbdf8-a2b3-431f-9474-d7e5f03c0f07 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:56.602087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fbdf8-a2b3-431f-9474-d7e5f03c0f07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:56.602278 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fbdf8-a2b3-431f-9474-d7e5f03c0f07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:56.613858 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fbdf8-a2b3-431f-9474-d7e5f03c0f07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:56.615414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fbdf8-a2b3-431f-9474-d7e5f03c0f07 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:56.627171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1074/2116] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:35:57.030303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba64df-1df5-44d8-a7e6-88d23c1e5365 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:57.033001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-65ba64df-1df5-44d8-a7e6-88d23c1e5365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:57.053323 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba64df-1df5-44d8-a7e6-88d23c1e5365 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:57.067897 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147f561-89fa-4bb7-9006-15b901270f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:57.069943 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147f561-89fa-4bb7-9006-15b901270f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:57.070234 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147f561-89fa-4bb7-9006-15b901270f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:57.070510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147f561-89fa-4bb7-9006-15b901270f72 admin admin] PATH_INFO: `/v3/projects/486d4d85b8b34a2f926210b6d088b33b` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:57.075395 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147f561-89fa-4bb7-9006-15b901270f72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=486d4d85b8b34a2f926210b6d088b33b)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:57.077559 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147f561-89fa-4bb7-9006-15b901270f72 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:57.175151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f147f561-89fa-4bb7-9006-15b901270f72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '486d4d85b8b34a2f926210b6d088b33b', 'request_id': 'req-f147f561-89fa-4bb7-9006-15b901270f72'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:57.188315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f147f561-89fa-4bb7-9006-15b901270f72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '486d4d85b8b34a2f926210b6d088b33b', 'request_id': 'req-f147f561-89fa-4bb7-9006-15b901270f72'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:57.190671 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:57.202624 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f147f561-89fa-4bb7-9006-15b901270f72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '486d4d85b8b34a2f926210b6d088b33b', 'request_id': 'req-f147f561-89fa-4bb7-9006-15b901270f72'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:57.222266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f147f561-89fa-4bb7-9006-15b901270f72 admin admin] The token cache is being invalidate because project 486d4d85b8b34a2f926210b6d088b33b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:35:57.222552 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f147f561-89fa-4bb7-9006-15b901270f72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:57.223367 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f147f561-89fa-4bb7-9006-15b901270f72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:57.225880 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1043/2117] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:35:57 2022] DELETE /identity/v3/projects/486d4d85b8b34a2f926210b6d088b33b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:57.233121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917803d-ddba-49a9-9e47-9e275f4a18b8 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:57.233121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917803d-ddba-49a9-9e47-9e275f4a18b8 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:57.233121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917803d-ddba-49a9-9e47-9e275f4a18b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:57.269123 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-b917803d-ddba-49a9-9e47-9e275f4a18b8 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:57.270837 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b917803d-ddba-49a9-9e47-9e275f4a18b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:57.385195 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1075/2118] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:57.394971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199898c-cb5f-4275-a9ee-2a2c21ee8088 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:57.395367 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199898c-cb5f-4275-a9ee-2a2c21ee8088 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:57.395719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199898c-cb5f-4275-a9ee-2a2c21ee8088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:57.423531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-2199898c-cb5f-4275-a9ee-2a2c21ee8088 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:57.425622 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2199898c-cb5f-4275-a9ee-2a2c21ee8088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:57.435528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1044/2119] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:57.440127 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917803d-ddba-49a9-9e47-9e275f4a18b8 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:57.463306 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917803d-ddba-49a9-9e47-9e275f4a18b8 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:57.469701 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced21811-a05a-4157-af7f-34fee6ca0c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:57.470510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced21811-a05a-4157-af7f-34fee6ca0c5d admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:57.470965 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced21811-a05a-4157-af7f-34fee6ca0c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:57.471308 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced21811-a05a-4157-af7f-34fee6ca0c5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:57.472122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced21811-a05a-4157-af7f-34fee6ca0c5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:35:57.472600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced21811-a05a-4157-af7f-34fee6ca0c5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:57.474571 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced21811-a05a-4157-af7f-34fee6ca0c5d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:57.485460 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:57.500127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199898c-cb5f-4275-a9ee-2a2c21ee8088 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:57.502062 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1076/2120] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:35:57.512581 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199898c-cb5f-4275-a9ee-2a2c21ee8088 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:57.518455 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27169b19-f206-4054-baaf-32355da80b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:57.519374 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27169b19-f206-4054-baaf-32355da80b4d admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:57.519820 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27169b19-f206-4054-baaf-32355da80b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:57.520134 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27169b19-f206-4054-baaf-32355da80b4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:57.520820 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27169b19-f206-4054-baaf-32355da80b4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:57.522792 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27169b19-f206-4054-baaf-32355da80b4d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:57.571532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1045/2121] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:35:57 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:57.572854 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced21811-a05a-4157-af7f-34fee6ca0c5d admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:57.584763 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced21811-a05a-4157-af7f-34fee6ca0c5d admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:57.699157 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa597e4-387c-4f0f-8cf9-b38fad6273a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:57.700869 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa597e4-387c-4f0f-8cf9-b38fad6273a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:57.701260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa597e4-387c-4f0f-8cf9-b38fad6273a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:57.701660 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa597e4-387c-4f0f-8cf9-b38fad6273a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:57.702476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa597e4-387c-4f0f-8cf9-b38fad6273a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:57.704752 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa597e4-387c-4f0f-8cf9-b38fad6273a6 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:57.778789 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1077/2122] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:35:57 2022] POST /identity/v3/users => generated 287 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:35:57.781647 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27169b19-f206-4054-baaf-32355da80b4d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:57.788943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa597e4-387c-4f0f-8cf9-b38fad6273a6 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:57.794229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2aa597e4-387c-4f0f-8cf9-b38fad6273a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:57.799243 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27169b19-f206-4054-baaf-32355da80b4d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:57.804589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ea410-5be0-4d58-bf69-648c0d0872ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:57.805390 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ea410-5be0-4d58-bf69-648c0d0872ca admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:57.805771 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ea410-5be0-4d58-bf69-648c0d0872ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:57.806222 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ea410-5be0-4d58-bf69-648c0d0872ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:57.807086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ea410-5be0-4d58-bf69-648c0d0872ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:57.810357 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ea410-5be0-4d58-bf69-648c0d0872ca admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:57.823100 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa597e4-387c-4f0f-8cf9-b38fad6273a6 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:57.828763 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1046/2123] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:35:57.829917 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ea410-5be0-4d58-bf69-648c0d0872ca admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:57.832794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e282632-2065-401f-8fba-28ea9bcd7b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:57.834852 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e282632-2065-401f-8fba-28ea9bcd7b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:57.836771 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e282632-2065-401f-8fba-28ea9bcd7b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:57.837525 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e282632-2065-401f-8fba-28ea9bcd7b3d admin admin] PATH_INFO: `/v3/users/e6c34715a6c54eaea82189e5150013a9` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:57.839382 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e282632-2065-401f-8fba-28ea9bcd7b3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6c34715a6c54eaea82189e5150013a9)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:57.841977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ea410-5be0-4d58-bf69-648c0d0872ca admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:57.842795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e282632-2065-401f-8fba-28ea9bcd7b3d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:57.849346 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6826f-bd23-4ab9-b8f4-60cfb5b97825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:57.850021 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6826f-bd23-4ab9-b8f4-60cfb5b97825 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:57.850305 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6826f-bd23-4ab9-b8f4-60cfb5b97825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:57.850591 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6826f-bd23-4ab9-b8f4-60cfb5b97825 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:57.851467 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6826f-bd23-4ab9-b8f4-60cfb5b97825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:57.854137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6826f-bd23-4ab9-b8f4-60cfb5b97825 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:57.873949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1047/2124] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:35:57.876977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6826f-bd23-4ab9-b8f4-60cfb5b97825 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:57.891858 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6826f-bd23-4ab9-b8f4-60cfb5b97825 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:57.898309 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c94bc-e7a4-4c28-8c96-dfedeea9f44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:57.899196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c94bc-e7a4-4c28-8c96-dfedeea9f44e admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:57.899740 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c94bc-e7a4-4c28-8c96-dfedeea9f44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:57.900238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c94bc-e7a4-4c28-8c96-dfedeea9f44e admin admin] PATH_INFO: `/v3/projects/63125418cd9949cdbf76099952b31950/users/6d903b4429664fceaa1344f1b99791dd/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:57.925148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c94bc-e7a4-4c28-8c96-dfedeea9f44e admin admin] RBAC: Authorizing `identity:create_grant(project_id=63125418cd9949cdbf76099952b31950, user_id=6d903b4429664fceaa1344f1b99791dd, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:57.928401 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c94bc-e7a4-4c28-8c96-dfedeea9f44e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:57.942244 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1e282632-2065-401f-8fba-28ea9bcd7b3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6c34715a6c54eaea82189e5150013a9'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:57.943291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1e282632-2065-401f-8fba-28ea9bcd7b3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6c34715a6c54eaea82189e5150013a9'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:57.946168 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1048/2125] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:35:57 2022] PUT /identity/v3/projects/63125418cd9949cdbf76099952b31950/users/6d903b4429664fceaa1344f1b99791dd/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:57.952246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d42d65-757f-4c9a-a21c-b417c7a959a7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:57.952608 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d42d65-757f-4c9a-a21c-b417c7a959a7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:57.952987 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d42d65-757f-4c9a-a21c-b417c7a959a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:57.957240 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1e282632-2065-401f-8fba-28ea9bcd7b3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6c34715a6c54eaea82189e5150013a9'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:57.975517 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1e282632-2065-401f-8fba-28ea9bcd7b3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6c34715a6c54eaea82189e5150013a9'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:57.985171 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d7d42d65-757f-4c9a-a21c-b417c7a959a7 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:57.985419 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1e282632-2065-401f-8fba-28ea9bcd7b3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6c34715a6c54eaea82189e5150013a9'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:57.986685 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d7d42d65-757f-4c9a-a21c-b417c7a959a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:57.993442 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1078/2126] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:35:57 2022] DELETE /identity/v3/users/e6c34715a6c54eaea82189e5150013a9 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:58.004459 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80ac50-5999-4edd-9dfc-fda59130375d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:58.004747 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80ac50-5999-4edd-9dfc-fda59130375d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:58.005036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80ac50-5999-4edd-9dfc-fda59130375d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:58.029632 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-df80ac50-5999-4edd-9dfc-fda59130375d None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:58.031180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-df80ac50-5999-4edd-9dfc-fda59130375d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:58.107201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1049/2127] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:58.119416 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d42d65-757f-4c9a-a21c-b417c7a959a7 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:58.121575 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d7d42d65-757f-4c9a-a21c-b417c7a959a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:58.185242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:58.214440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1079/2128] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:58.228002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80ac50-5999-4edd-9dfc-fda59130375d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:58.230420 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-df80ac50-5999-4edd-9dfc-fda59130375d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:58.264522 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d42d65-757f-4c9a-a21c-b417c7a959a7 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:58.309306 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:58.332161 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80ac50-5999-4edd-9dfc-fda59130375d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:58.337520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2188d-a414-416b-a2aa-82ea7e0c32e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:58.338771 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2188d-a414-416b-a2aa-82ea7e0c32e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:58.339384 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2188d-a414-416b-a2aa-82ea7e0c32e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:58.339929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2188d-a414-416b-a2aa-82ea7e0c32e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:58.345078 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168c6d5-9ab9-41ea-808b-ed0ab9335c09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:58.346090 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168c6d5-9ab9-41ea-808b-ed0ab9335c09 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:58.346537 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168c6d5-9ab9-41ea-808b-ed0ab9335c09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:58.346825 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168c6d5-9ab9-41ea-808b-ed0ab9335c09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:58.353227 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2188d-a414-416b-a2aa-82ea7e0c32e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:58.355123 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2188d-a414-416b-a2aa-82ea7e0c32e3 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:58.358933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168c6d5-9ab9-41ea-808b-ed0ab9335c09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:58.360527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168c6d5-9ab9-41ea-808b-ed0ab9335c09 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:58.367848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1080/2129] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:35:58.378436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1050/2130] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:35:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:35:58.932821 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2188d-a414-416b-a2aa-82ea7e0c32e3 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:58.935052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ffe2188d-a414-416b-a2aa-82ea7e0c32e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:58.962995 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2188d-a414-416b-a2aa-82ea7e0c32e3 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:58.970456 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e4712-e98a-4e3f-983a-3ccc114b72d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:58.973186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e4712-e98a-4e3f-983a-3ccc114b72d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:58.974182 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e4712-e98a-4e3f-983a-3ccc114b72d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:58.974379 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e4712-e98a-4e3f-983a-3ccc114b72d4 admin admin] PATH_INFO: `/v3/projects/b0824c1b01aa41288e86f341565c5409` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:58.983173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e4712-e98a-4e3f-983a-3ccc114b72d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0824c1b01aa41288e86f341565c5409)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:58.985209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e4712-e98a-4e3f-983a-3ccc114b72d4 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:59.073589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-819e4712-e98a-4e3f-983a-3ccc114b72d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0824c1b01aa41288e86f341565c5409', 'request_id': 'req-819e4712-e98a-4e3f-983a-3ccc114b72d4'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:59.091911 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-819e4712-e98a-4e3f-983a-3ccc114b72d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0824c1b01aa41288e86f341565c5409', 'request_id': 'req-819e4712-e98a-4e3f-983a-3ccc114b72d4'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:59.114200 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-819e4712-e98a-4e3f-983a-3ccc114b72d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0824c1b01aa41288e86f341565c5409', 'request_id': 'req-819e4712-e98a-4e3f-983a-3ccc114b72d4'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:59.146267 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-819e4712-e98a-4e3f-983a-3ccc114b72d4 admin admin] The token cache is being invalidate because project b0824c1b01aa41288e86f341565c5409 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:35:59.146904 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-819e4712-e98a-4e3f-983a-3ccc114b72d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:59.148298 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-819e4712-e98a-4e3f-983a-3ccc114b72d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:59.154242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1081/2131] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:35:58 2022] DELETE /identity/v3/projects/b0824c1b01aa41288e86f341565c5409 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:59.156592 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168c6d5-9ab9-41ea-808b-ed0ab9335c09 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:59.173643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168c6d5-9ab9-41ea-808b-ed0ab9335c09 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:59.271715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052885f-cd59-4bad-acf4-259336ba0f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:59.281102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052885f-cd59-4bad-acf4-259336ba0f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:59.281530 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052885f-cd59-4bad-acf4-259336ba0f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:59.281779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052885f-cd59-4bad-acf4-259336ba0f91 admin admin] PATH_INFO: `/v3/users/2ec954936a0b48ca874b064af1ebfee3` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:59.283294 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052885f-cd59-4bad-acf4-259336ba0f91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ec954936a0b48ca874b064af1ebfee3)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:59.285353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052885f-cd59-4bad-acf4-259336ba0f91 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:59.381138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4052885f-cd59-4bad-acf4-259336ba0f91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ec954936a0b48ca874b064af1ebfee3'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:59.381829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4052885f-cd59-4bad-acf4-259336ba0f91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ec954936a0b48ca874b064af1ebfee3'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:59.404141 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4052885f-cd59-4bad-acf4-259336ba0f91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ec954936a0b48ca874b064af1ebfee3'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:59.420731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4052885f-cd59-4bad-acf4-259336ba0f91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ec954936a0b48ca874b064af1ebfee3'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:59.421491 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4052885f-cd59-4bad-acf4-259336ba0f91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ec954936a0b48ca874b064af1ebfee3'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:59.439019 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1051/2132] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:35:59 2022] DELETE /identity/v3/users/2ec954936a0b48ca874b064af1ebfee3 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:00.258103 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e4712-e98a-4e3f-983a-3ccc114b72d4 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:00.259920 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-819e4712-e98a-4e3f-983a-3ccc114b72d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:00.313072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e4712-e98a-4e3f-983a-3ccc114b72d4 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:00.550764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da0302-344b-4390-9747-4f1896f98354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:00.551710 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da0302-344b-4390-9747-4f1896f98354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:00.552195 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da0302-344b-4390-9747-4f1896f98354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:00.552701 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da0302-344b-4390-9747-4f1896f98354 admin admin] PATH_INFO: `/v3/projects/27b70249fbb641f3b38bd6860f9c73af` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:00.575977 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da0302-344b-4390-9747-4f1896f98354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27b70249fbb641f3b38bd6860f9c73af)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:00.580082 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da0302-344b-4390-9747-4f1896f98354 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:00.649514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:00.704500 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a9da0302-344b-4390-9747-4f1896f98354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27b70249fbb641f3b38bd6860f9c73af', 'request_id': 'req-a9da0302-344b-4390-9747-4f1896f98354'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:00.720440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a9da0302-344b-4390-9747-4f1896f98354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27b70249fbb641f3b38bd6860f9c73af', 'request_id': 'req-a9da0302-344b-4390-9747-4f1896f98354'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:00.731413 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a9da0302-344b-4390-9747-4f1896f98354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27b70249fbb641f3b38bd6860f9c73af', 'request_id': 'req-a9da0302-344b-4390-9747-4f1896f98354'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:00.757443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a9da0302-344b-4390-9747-4f1896f98354 admin admin] The token cache is being invalidate because project 27b70249fbb641f3b38bd6860f9c73af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:36:00.757655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a9da0302-344b-4390-9747-4f1896f98354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:00.758473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a9da0302-344b-4390-9747-4f1896f98354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:00.761339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1082/2133] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:36:00 2022] DELETE /identity/v3/projects/27b70249fbb641f3b38bd6860f9c73af => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:00.766282 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052885f-cd59-4bad-acf4-259336ba0f91 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:00.775060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4052885f-cd59-4bad-acf4-259336ba0f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:00.816041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052885f-cd59-4bad-acf4-259336ba0f91 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:00.915722 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:00.948527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a556d7b7-fb7f-40e8-8372-9acdec14e995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:00.950357 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556d7b7-fb7f-40e8-8372-9acdec14e995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:00.951163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556d7b7-fb7f-40e8-8372-9acdec14e995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:00.951684 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556d7b7-fb7f-40e8-8372-9acdec14e995 admin admin] PATH_INFO: `/v3/users/3e1680179f0b478ca987ed69959c1636` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:00.953460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a556d7b7-fb7f-40e8-8372-9acdec14e995 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e1680179f0b478ca987ed69959c1636)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:00.955690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a556d7b7-fb7f-40e8-8372-9acdec14e995 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:01.094721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a556d7b7-fb7f-40e8-8372-9acdec14e995 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e1680179f0b478ca987ed69959c1636'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:01.097129 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a556d7b7-fb7f-40e8-8372-9acdec14e995 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e1680179f0b478ca987ed69959c1636'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:01.113080 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a556d7b7-fb7f-40e8-8372-9acdec14e995 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e1680179f0b478ca987ed69959c1636'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:01.137494 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a556d7b7-fb7f-40e8-8372-9acdec14e995 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e1680179f0b478ca987ed69959c1636'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:01.138618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a556d7b7-fb7f-40e8-8372-9acdec14e995 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e1680179f0b478ca987ed69959c1636'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:01.170375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1052/2134] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:36:00 2022] DELETE /identity/v3/users/3e1680179f0b478ca987ed69959c1636 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:02.434710 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da0302-344b-4390-9747-4f1896f98354 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:02.438677 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a9da0302-344b-4390-9747-4f1896f98354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:02.474056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da0302-344b-4390-9747-4f1896f98354 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:02.600479 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08a9f4-66a6-49b4-92fc-3f1373231298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:02.602712 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08a9f4-66a6-49b4-92fc-3f1373231298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:02.603533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08a9f4-66a6-49b4-92fc-3f1373231298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:02.604082 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08a9f4-66a6-49b4-92fc-3f1373231298 admin admin] PATH_INFO: `/v3/projects/9c1202d7ef6f4ae491f30971e23efe28` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:02.607242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08a9f4-66a6-49b4-92fc-3f1373231298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c1202d7ef6f4ae491f30971e23efe28)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:02.612459 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08a9f4-66a6-49b4-92fc-3f1373231298 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:02.739844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0e08a9f4-66a6-49b4-92fc-3f1373231298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c1202d7ef6f4ae491f30971e23efe28', 'request_id': 'req-0e08a9f4-66a6-49b4-92fc-3f1373231298'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:02.748029 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0e08a9f4-66a6-49b4-92fc-3f1373231298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c1202d7ef6f4ae491f30971e23efe28', 'request_id': 'req-0e08a9f4-66a6-49b4-92fc-3f1373231298'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:02.757908 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0e08a9f4-66a6-49b4-92fc-3f1373231298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c1202d7ef6f4ae491f30971e23efe28', 'request_id': 'req-0e08a9f4-66a6-49b4-92fc-3f1373231298'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:02.775609 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0e08a9f4-66a6-49b4-92fc-3f1373231298 admin admin] The token cache is being invalidate because project 9c1202d7ef6f4ae491f30971e23efe28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:36:02.775979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0e08a9f4-66a6-49b4-92fc-3f1373231298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:02.776744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0e08a9f4-66a6-49b4-92fc-3f1373231298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:02.779222 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1083/2135] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:36:02 2022] DELETE /identity/v3/projects/9c1202d7ef6f4ae491f30971e23efe28 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:36:02.785465 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c94693-37ff-4d37-8444-1bd672546b1b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:02.786022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c94693-37ff-4d37-8444-1bd672546b1b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:02.786137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c94693-37ff-4d37-8444-1bd672546b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:02.882624 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d8c94693-37ff-4d37-8444-1bd672546b1b None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:02.884285 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d8c94693-37ff-4d37-8444-1bd672546b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:03.087635 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1053/2136] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:03.097446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26cda10-8005-4d98-8825-360c042569b5 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:03.097772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26cda10-8005-4d98-8825-360c042569b5 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:03.098769 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26cda10-8005-4d98-8825-360c042569b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:03.129851 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a26cda10-8005-4d98-8825-360c042569b5 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:03.131593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a26cda10-8005-4d98-8825-360c042569b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:03.151002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1084/2137] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:03.156052 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c94693-37ff-4d37-8444-1bd672546b1b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:03.180998 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c94693-37ff-4d37-8444-1bd672546b1b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:03.208751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf12fb6-736b-45a7-b5d8-573aa89a7dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:03.212590 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf12fb6-736b-45a7-b5d8-573aa89a7dcb admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:03.214449 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf12fb6-736b-45a7-b5d8-573aa89a7dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:03.215941 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf12fb6-736b-45a7-b5d8-573aa89a7dcb admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:03.219061 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf12fb6-736b-45a7-b5d8-573aa89a7dcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:36:03.219396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf12fb6-736b-45a7-b5d8-573aa89a7dcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:03.228393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf12fb6-736b-45a7-b5d8-573aa89a7dcb admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:03.326171 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1054/2138] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:36:03.330737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26cda10-8005-4d98-8825-360c042569b5 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:03.348315 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26cda10-8005-4d98-8825-360c042569b5 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:03.353213 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a51d68-57b8-4bd0-88d2-3c660d421b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:03.354046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a51d68-57b8-4bd0-88d2-3c660d421b7d admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:03.354308 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a51d68-57b8-4bd0-88d2-3c660d421b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:03.354602 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a51d68-57b8-4bd0-88d2-3c660d421b7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:03.355407 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a51d68-57b8-4bd0-88d2-3c660d421b7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:03.358439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a51d68-57b8-4bd0-88d2-3c660d421b7d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:03.399625 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1085/2139] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:36:03 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:36:03.404592 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf12fb6-736b-45a7-b5d8-573aa89a7dcb admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:03.429258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf12fb6-736b-45a7-b5d8-573aa89a7dcb admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:03.565269 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:03.632088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeecaea-a912-4a9e-a906-4cfe33b40346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:03.632782 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeecaea-a912-4a9e-a906-4cfe33b40346 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:03.633092 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeecaea-a912-4a9e-a906-4cfe33b40346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:03.633338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeecaea-a912-4a9e-a906-4cfe33b40346 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:03.646148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeecaea-a912-4a9e-a906-4cfe33b40346 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:03.670073 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeecaea-a912-4a9e-a906-4cfe33b40346 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:03.811854 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a51d68-57b8-4bd0-88d2-3c660d421b7d admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:03.833466 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1055/2140] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:36:03 2022] POST /identity/v3/users => generated 293 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:36:03.840955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a51d68-57b8-4bd0-88d2-3c660d421b7d admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:03.855207 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059a259-a2c3-4447-a906-306b4f49456b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:03.856403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059a259-a2c3-4447-a906-306b4f49456b admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:03.857314 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059a259-a2c3-4447-a906-306b4f49456b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:03.858826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059a259-a2c3-4447-a906-306b4f49456b admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:03.861028 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059a259-a2c3-4447-a906-306b4f49456b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:03.864562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059a259-a2c3-4447-a906-306b4f49456b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:03.902192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1086/2141] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:36:03.903321 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeecaea-a912-4a9e-a906-4cfe33b40346 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:03.920688 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeecaea-a912-4a9e-a906-4cfe33b40346 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:03.942679 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc809fe-c4d2-49f8-97db-3c22c92f011c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:03.943772 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc809fe-c4d2-49f8-97db-3c22c92f011c admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:03.944276 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc809fe-c4d2-49f8-97db-3c22c92f011c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:03.944759 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc809fe-c4d2-49f8-97db-3c22c92f011c admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:03.945924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc809fe-c4d2-49f8-97db-3c22c92f011c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:03.948009 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc809fe-c4d2-49f8-97db-3c22c92f011c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:03.986013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1056/2142] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:36:03.990479 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059a259-a2c3-4447-a906-306b4f49456b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:04.011256 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:04.036659 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059a259-a2c3-4447-a906-306b4f49456b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:04.068237 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778d9b2-e5c0-4ade-8657-7d8cb96550ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:04.071910 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778d9b2-e5c0-4ade-8657-7d8cb96550ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:04.072227 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778d9b2-e5c0-4ade-8657-7d8cb96550ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:04.072615 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778d9b2-e5c0-4ade-8657-7d8cb96550ef admin admin] PATH_INFO: `/v3/projects/b151bad1a9184717b83bd6726d169504/users/eef8a4a166d149f5a9f34e318058c1de/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:04.112348 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2778d9b2-e5c0-4ade-8657-7d8cb96550ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=b151bad1a9184717b83bd6726d169504, user_id=eef8a4a166d149f5a9f34e318058c1de, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:04.116936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2778d9b2-e5c0-4ade-8657-7d8cb96550ef admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:04.162098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1087/2143] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:36:03 2022] PUT /identity/v3/projects/b151bad1a9184717b83bd6726d169504/users/eef8a4a166d149f5a9f34e318058c1de/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:04.170725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3bead8-b25a-4276-a292-2b411460cd64 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:04.171207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3bead8-b25a-4276-a292-2b411460cd64 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:04.171697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3bead8-b25a-4276-a292-2b411460cd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:04.241390 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7a3bead8-b25a-4276-a292-2b411460cd64 None None] MFA Rules not processed for user `eef8a4a166d149f5a9f34e318058c1de`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:04.244644 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7a3bead8-b25a-4276-a292-2b411460cd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:04.408150 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1057/2144] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:04.413269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778d9b2-e5c0-4ade-8657-7d8cb96550ef admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:04.541316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778d9b2-e5c0-4ade-8657-7d8cb96550ef admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:04.549309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8898b-5cba-4bb1-b9cf-73e2bcabd874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:04.550840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8898b-5cba-4bb1-b9cf-73e2bcabd874 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:04.551500 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8898b-5cba-4bb1-b9cf-73e2bcabd874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:04.552277 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8898b-5cba-4bb1-b9cf-73e2bcabd874 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:04.553469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8898b-5cba-4bb1-b9cf-73e2bcabd874 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:04.556593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8898b-5cba-4bb1-b9cf-73e2bcabd874 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:04.620893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1088/2145] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:36:04 2022] POST /identity/v3/projects => generated 383 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:36:04.625907 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3bead8-b25a-4276-a292-2b411460cd64 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:04.649597 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3bead8-b25a-4276-a292-2b411460cd64 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:04.840519 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33894207-8ebb-4b93-bea1-9870a6f176f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:04.842026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33894207-8ebb-4b93-bea1-9870a6f176f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:04.842369 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33894207-8ebb-4b93-bea1-9870a6f176f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:04.842975 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33894207-8ebb-4b93-bea1-9870a6f176f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:04.844966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33894207-8ebb-4b93-bea1-9870a6f176f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:04.869172 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33894207-8ebb-4b93-bea1-9870a6f176f1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:04.929501 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1058/2146] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:36:04 2022] POST /identity/v3/users => generated 292 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:36:04.932933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8898b-5cba-4bb1-b9cf-73e2bcabd874 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:04.949414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8898b-5cba-4bb1-b9cf-73e2bcabd874 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:04.962966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e9a89-60c5-41f2-9283-79cd891b9f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:04.965499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e9a89-60c5-41f2-9283-79cd891b9f11 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:04.967015 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e9a89-60c5-41f2-9283-79cd891b9f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:04.969296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e9a89-60c5-41f2-9283-79cd891b9f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:04.970325 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e9a89-60c5-41f2-9283-79cd891b9f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:04.982168 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e9a89-60c5-41f2-9283-79cd891b9f11 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:05.003135 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1089/2147] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:36:05.008883 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33894207-8ebb-4b93-bea1-9870a6f176f1 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:05.033636 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33894207-8ebb-4b93-bea1-9870a6f176f1 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:05.049757 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00709338-26a9-4471-90cd-65ae52b09f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:05.051063 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00709338-26a9-4471-90cd-65ae52b09f7d admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:05.051552 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00709338-26a9-4471-90cd-65ae52b09f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:05.052026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00709338-26a9-4471-90cd-65ae52b09f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:05.053147 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00709338-26a9-4471-90cd-65ae52b09f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:05.055352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00709338-26a9-4471-90cd-65ae52b09f7d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:05.096578 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1059/2148] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:36:05.096853 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e9a89-60c5-41f2-9283-79cd891b9f11 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:05.116410 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e9a89-60c5-41f2-9283-79cd891b9f11 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:05.132940 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f22100-e69d-43e7-b20a-0f6f17c5a41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:05.133872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f22100-e69d-43e7-b20a-0f6f17c5a41a admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:05.134320 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f22100-e69d-43e7-b20a-0f6f17c5a41a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:05.134882 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f22100-e69d-43e7-b20a-0f6f17c5a41a admin admin] PATH_INFO: `/v3/projects/12b3a794f28249079677cd993fcc7618/users/2f0ccc26b642456992521502e04966d5/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:05.178342 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f22100-e69d-43e7-b20a-0f6f17c5a41a admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b3a794f28249079677cd993fcc7618, user_id=2f0ccc26b642456992521502e04966d5, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:05.187071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f22100-e69d-43e7-b20a-0f6f17c5a41a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:05.250129 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1090/2149] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:36:05 2022] PUT /identity/v3/projects/12b3a794f28249079677cd993fcc7618/users/2f0ccc26b642456992521502e04966d5/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:36:05.268160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81d36e-edf8-48f4-9d71-7e8772928d7e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:05.268762 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81d36e-edf8-48f4-9d71-7e8772928d7e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:05.269262 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81d36e-edf8-48f4-9d71-7e8772928d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:05.344923 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-5c81d36e-edf8-48f4-9d71-7e8772928d7e None None] MFA Rules not processed for user `2f0ccc26b642456992521502e04966d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:05.346395 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5c81d36e-edf8-48f4-9d71-7e8772928d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:05.462996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:05.574213 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1060/2150] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 01 15:36:05.580181 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f22100-e69d-43e7-b20a-0f6f17c5a41a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:05.720406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f22100-e69d-43e7-b20a-0f6f17c5a41a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:05.726269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012918e-778c-4cf9-9a84-cde5fafeeab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:05.727235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012918e-778c-4cf9-9a84-cde5fafeeab0 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:05.727785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012918e-778c-4cf9-9a84-cde5fafeeab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:05.728227 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012918e-778c-4cf9-9a84-cde5fafeeab0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:05.729093 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012918e-778c-4cf9-9a84-cde5fafeeab0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:05.732719 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012918e-778c-4cf9-9a84-cde5fafeeab0 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:05.801557 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81d36e-edf8-48f4-9d71-7e8772928d7e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:05.805043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1091/2151] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:36:05 2022] POST /identity/v3/projects => generated 383 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:36:05.819216 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81d36e-edf8-48f4-9d71-7e8772928d7e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:06.046393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a8fec-3022-4e57-8f30-c08112b290de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:06.047980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a8fec-3022-4e57-8f30-c08112b290de admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:06.050352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a8fec-3022-4e57-8f30-c08112b290de admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:06.050883 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a8fec-3022-4e57-8f30-c08112b290de admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:06.053860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a8fec-3022-4e57-8f30-c08112b290de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:06.056184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a8fec-3022-4e57-8f30-c08112b290de admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:06.130753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1061/2152] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:36:05 2022] POST /identity/v3/users => generated 298 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:36:06.137973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012918e-778c-4cf9-9a84-cde5fafeeab0 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:06.182850 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012918e-778c-4cf9-9a84-cde5fafeeab0 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:06.188899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69baf6-b661-4a3a-b42d-770bbce3b193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:06.189959 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69baf6-b661-4a3a-b42d-770bbce3b193 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:06.190422 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69baf6-b661-4a3a-b42d-770bbce3b193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:06.190840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69baf6-b661-4a3a-b42d-770bbce3b193 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:06.191695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e69baf6-b661-4a3a-b42d-770bbce3b193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:06.193670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e69baf6-b661-4a3a-b42d-770bbce3b193 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:06.223057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1092/2153] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:36:06.228300 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a8fec-3022-4e57-8f30-c08112b290de admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:06.258837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a8fec-3022-4e57-8f30-c08112b290de admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:06.278255 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d8dbe-2c14-4476-a8c5-e9a356d683bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:06.279363 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d8dbe-2c14-4476-a8c5-e9a356d683bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:06.279884 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d8dbe-2c14-4476-a8c5-e9a356d683bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:06.280353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d8dbe-2c14-4476-a8c5-e9a356d683bb admin admin] PATH_INFO: `/v3/projects/d919a8c5075a483fa8a73d5ff54e0746/users/00d4965e5270403d91754e47e58d9eb7/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:06.327315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d8dbe-2c14-4476-a8c5-e9a356d683bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d919a8c5075a483fa8a73d5ff54e0746, user_id=00d4965e5270403d91754e47e58d9eb7, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:06.331190 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d8dbe-2c14-4476-a8c5-e9a356d683bb admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:06.358341 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1062/2154] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:36:06 2022] PUT /identity/v3/projects/d919a8c5075a483fa8a73d5ff54e0746/users/00d4965e5270403d91754e47e58d9eb7/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:06.362997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69baf6-b661-4a3a-b42d-770bbce3b193 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:06.377098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:06.386126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69baf6-b661-4a3a-b42d-770bbce3b193 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:06.567282 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2a7b9-4a3a-4ef6-b0be-6fdf4cb96feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:06.569297 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2a7b9-4a3a-4ef6-b0be-6fdf4cb96feb admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:06.569816 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2a7b9-4a3a-4ef6-b0be-6fdf4cb96feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:06.571482 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2a7b9-4a3a-4ef6-b0be-6fdf4cb96feb admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:06.572787 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec2a7b9-4a3a-4ef6-b0be-6fdf4cb96feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:06.578903 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec2a7b9-4a3a-4ef6-b0be-6fdf4cb96feb admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:06.616735 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1093/2155] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:36:06.619082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d8dbe-2c14-4476-a8c5-e9a356d683bb admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:06.694838 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d8dbe-2c14-4476-a8c5-e9a356d683bb admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:06.737449 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818cb3b3-c84a-468f-9c45-76e45969ea1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:06.742989 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cb3b3-c84a-468f-9c45-76e45969ea1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:06.749402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cb3b3-c84a-468f-9c45-76e45969ea1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:06.749959 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cb3b3-c84a-468f-9c45-76e45969ea1e admin admin] PATH_INFO: `/v3/projects/d919a8c5075a483fa8a73d5ff54e0746/users/00d4965e5270403d91754e47e58d9eb7/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:06.754553 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818cb3b3-c84a-468f-9c45-76e45969ea1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d919a8c5075a483fa8a73d5ff54e0746, user_id=00d4965e5270403d91754e47e58d9eb7, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:06.793861 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818cb3b3-c84a-468f-9c45-76e45969ea1e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:06.864668 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1063/2156] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:36:06 2022] PUT /identity/v3/projects/d919a8c5075a483fa8a73d5ff54e0746/users/00d4965e5270403d91754e47e58d9eb7/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:36:06.867700 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a2aa5-e516-403b-82a7-ba850ce73afe None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:06.868288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a2aa5-e516-403b-82a7-ba850ce73afe None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:06.868721 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a2aa5-e516-403b-82a7-ba850ce73afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:06.950919 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8c0a2aa5-e516-403b-82a7-ba850ce73afe None None] MFA Rules not processed for user `00d4965e5270403d91754e47e58d9eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:06.953350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8c0a2aa5-e516-403b-82a7-ba850ce73afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:07.189915 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1094/2157] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:07.212728 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cb7bf-8f09-4b5e-aa36-aba9616af4c3 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:07.212728 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cb7bf-8f09-4b5e-aa36-aba9616af4c3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:07.212728 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cb7bf-8f09-4b5e-aa36-aba9616af4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:07.213439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a2aa5-e516-403b-82a7-ba850ce73afe None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:07.213815 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8c0a2aa5-e516-403b-82a7-ba850ce73afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:07.306156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-877cb7bf-8f09-4b5e-aa36-aba9616af4c3 None None] MFA Rules not processed for user `6d903b4429664fceaa1344f1b99791dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:07.307422 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-877cb7bf-8f09-4b5e-aa36-aba9616af4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:07.329737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a2aa5-e516-403b-82a7-ba850ce73afe None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:07.334364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064fc32-860a-4cac-bfcf-704df9ad9a83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:07.335176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064fc32-860a-4cac-bfcf-704df9ad9a83 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:07.335725 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064fc32-860a-4cac-bfcf-704df9ad9a83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:07.336476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064fc32-860a-4cac-bfcf-704df9ad9a83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:07.494093 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:07.510507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1064/2158] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:07.513010 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cb7bf-8f09-4b5e-aa36-aba9616af4c3 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:07.521708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-877cb7bf-8f09-4b5e-aa36-aba9616af4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:07.536873 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064fc32-860a-4cac-bfcf-704df9ad9a83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:07.538631 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064fc32-860a-4cac-bfcf-704df9ad9a83 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:07.550637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:07.550873 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1095/2159] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:36:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:36:07.739401 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cb7bf-8f09-4b5e-aa36-aba9616af4c3 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:07.764229 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f6e59-91ef-443f-a581-621f6145075a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:07.767545 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f6e59-91ef-443f-a581-621f6145075a admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:07.768027 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f6e59-91ef-443f-a581-621f6145075a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:07.768549 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f6e59-91ef-443f-a581-621f6145075a admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:07.772351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f6e59-91ef-443f-a581-621f6145075a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:07.781321 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f6e59-91ef-443f-a581-621f6145075a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:07.867963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1065/2160] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:36:07 2022] POST /identity/v3/projects => generated 369 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:36:07.869218 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064fc32-860a-4cac-bfcf-704df9ad9a83 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:07.888682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064fc32-860a-4cac-bfcf-704df9ad9a83 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:08.054896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e9a7d-4ef6-43d2-a8f9-b168d105b389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:08.055590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e9a7d-4ef6-43d2-a8f9-b168d105b389 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:08.055876 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e9a7d-4ef6-43d2-a8f9-b168d105b389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:08.056148 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e9a7d-4ef6-43d2-a8f9-b168d105b389 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:08.056838 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e9a7d-4ef6-43d2-a8f9-b168d105b389 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:08.059437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e9a7d-4ef6-43d2-a8f9-b168d105b389 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:08.102977 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1096/2161] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:36:07 2022] POST /identity/v3/users => generated 291 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:36:08.108665 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f6e59-91ef-443f-a581-621f6145075a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:08.131170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f6e59-91ef-443f-a581-621f6145075a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:08.139303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd17b1-bb75-4408-b8c1-01829b0a5736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:08.140651 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd17b1-bb75-4408-b8c1-01829b0a5736 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:08.141000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd17b1-bb75-4408-b8c1-01829b0a5736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:08.142157 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd17b1-bb75-4408-b8c1-01829b0a5736 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:08.142668 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd17b1-bb75-4408-b8c1-01829b0a5736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:08.145345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd17b1-bb75-4408-b8c1-01829b0a5736 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:08.176317 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1066/2162] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:36:08.181199 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e9a7d-4ef6-43d2-a8f9-b168d105b389 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:08.208105 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e9a7d-4ef6-43d2-a8f9-b168d105b389 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:08.220362 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e71a3e-5f4a-49cb-84f1-b765da2bdaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:08.222746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e71a3e-5f4a-49cb-84f1-b765da2bdaf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:08.222994 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e71a3e-5f4a-49cb-84f1-b765da2bdaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:08.223281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e71a3e-5f4a-49cb-84f1-b765da2bdaf8 admin admin] PATH_INFO: `/v3/projects/935097df97b74d04aa0603250d968f5c/users/23e07f301bb848188defaddfa9db59b0/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:08.264945 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e71a3e-5f4a-49cb-84f1-b765da2bdaf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=935097df97b74d04aa0603250d968f5c, user_id=23e07f301bb848188defaddfa9db59b0, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:08.264945 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e71a3e-5f4a-49cb-84f1-b765da2bdaf8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:08.297747 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1097/2163] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:36:08 2022] PUT /identity/v3/projects/935097df97b74d04aa0603250d968f5c/users/23e07f301bb848188defaddfa9db59b0/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:08.302573 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd17b1-bb75-4408-b8c1-01829b0a5736 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:08.335289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd17b1-bb75-4408-b8c1-01829b0a5736 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:08.511186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba56e4-78dd-4fa2-83d6-06182d8dc0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:08.516210 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba56e4-78dd-4fa2-83d6-06182d8dc0f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:08.516706 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba56e4-78dd-4fa2-83d6-06182d8dc0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:08.517636 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba56e4-78dd-4fa2-83d6-06182d8dc0f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:08.518249 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba56e4-78dd-4fa2-83d6-06182d8dc0f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:08.520044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba56e4-78dd-4fa2-83d6-06182d8dc0f1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:08.568331 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1067/2164] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:36:08.573215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e71a3e-5f4a-49cb-84f1-b765da2bdaf8 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:08.593062 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e71a3e-5f4a-49cb-84f1-b765da2bdaf8 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:08.609777 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aad555-fd95-49c8-8ae2-6a2f1413c4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:08.610702 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aad555-fd95-49c8-8ae2-6a2f1413c4dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:08.611095 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aad555-fd95-49c8-8ae2-6a2f1413c4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:08.611484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aad555-fd95-49c8-8ae2-6a2f1413c4dd admin admin] PATH_INFO: `/v3/projects/935097df97b74d04aa0603250d968f5c/users/23e07f301bb848188defaddfa9db59b0/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:08.615924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aad555-fd95-49c8-8ae2-6a2f1413c4dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=935097df97b74d04aa0603250d968f5c, user_id=23e07f301bb848188defaddfa9db59b0, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:08.622690 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aad555-fd95-49c8-8ae2-6a2f1413c4dd admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:08.663608 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1098/2165] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:36:08 2022] PUT /identity/v3/projects/935097df97b74d04aa0603250d968f5c/users/23e07f301bb848188defaddfa9db59b0/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:36:09.525601 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba56e4-78dd-4fa2-83d6-06182d8dc0f1 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:09.674270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba56e4-78dd-4fa2-83d6-06182d8dc0f1 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:09.783654 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:09.804414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298bd17-6cc6-42ed-8937-eb6c4ace206a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:09.805486 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298bd17-6cc6-42ed-8937-eb6c4ace206a service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:09.806888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298bd17-6cc6-42ed-8937-eb6c4ace206a service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:09.819737 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298bd17-6cc6-42ed-8937-eb6c4ace206a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:09.948362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298bd17-6cc6-42ed-8937-eb6c4ace206a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:09.951173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298bd17-6cc6-42ed-8937-eb6c4ace206a service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:09.965022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1068/2166] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:36:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:36:11.227968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aad555-fd95-49c8-8ae2-6a2f1413c4dd admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:11.231058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-05aad555-fd95-49c8-8ae2-6a2f1413c4dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:11.255034 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aad555-fd95-49c8-8ae2-6a2f1413c4dd admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:11.268198 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813513df-dfb2-4469-9aa3-18875f1ac10d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:11.271023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813513df-dfb2-4469-9aa3-18875f1ac10d service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:11.272655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813513df-dfb2-4469-9aa3-18875f1ac10d service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:11.275402 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813513df-dfb2-4469-9aa3-18875f1ac10d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:11.503202 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813513df-dfb2-4469-9aa3-18875f1ac10d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:11.508189 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813513df-dfb2-4469-9aa3-18875f1ac10d service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:11.573253 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1099/2167] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:36:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:36:17.157477 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298bd17-6cc6-42ed-8937-eb6c4ace206a service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:17.322895 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298bd17-6cc6-42ed-8937-eb6c4ace206a service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:17.337692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455dc3a-3853-4f11-a867-27a76249043f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:17.338942 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455dc3a-3853-4f11-a867-27a76249043f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:17.339529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455dc3a-3853-4f11-a867-27a76249043f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:17.340112 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455dc3a-3853-4f11-a867-27a76249043f admin admin] PATH_INFO: `/v3/users/eef8a4a166d149f5a9f34e318058c1de` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:17.343604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455dc3a-3853-4f11-a867-27a76249043f admin admin] RBAC: Authorizing `identity:delete_user(user_id=eef8a4a166d149f5a9f34e318058c1de)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:17.347602 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455dc3a-3853-4f11-a867-27a76249043f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:17.483178 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9455dc3a-3853-4f11-a867-27a76249043f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eef8a4a166d149f5a9f34e318058c1de'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:17.484305 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9455dc3a-3853-4f11-a867-27a76249043f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eef8a4a166d149f5a9f34e318058c1de'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:17.499135 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9455dc3a-3853-4f11-a867-27a76249043f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eef8a4a166d149f5a9f34e318058c1de'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:17.545461 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9455dc3a-3853-4f11-a867-27a76249043f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eef8a4a166d149f5a9f34e318058c1de'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:17.549713 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9455dc3a-3853-4f11-a867-27a76249043f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eef8a4a166d149f5a9f34e318058c1de'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:17.610870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1069/2168] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:36:17 2022] DELETE /identity/v3/users/eef8a4a166d149f5a9f34e318058c1de => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:17.613357 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8579df5-c566-4d70-8d2c-528950706743 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:17.615217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8579df5-c566-4d70-8d2c-528950706743 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:17.615741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8579df5-c566-4d70-8d2c-528950706743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:17.690539 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f8579df5-c566-4d70-8d2c-528950706743 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:17.692120 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f8579df5-c566-4d70-8d2c-528950706743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:17.800735 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:17.941218 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1100/2169] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:17.969439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455dc3a-3853-4f11-a867-27a76249043f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:17.971375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9455dc3a-3853-4f11-a867-27a76249043f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:18.015526 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:18.143972 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455dc3a-3853-4f11-a867-27a76249043f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:18.154740 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661a5c5-4d30-410f-a929-9914f5d3be77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:18.155417 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661a5c5-4d30-410f-a929-9914f5d3be77 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:18.155709 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661a5c5-4d30-410f-a929-9914f5d3be77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:18.155984 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661a5c5-4d30-410f-a929-9914f5d3be77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:18.172858 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661a5c5-4d30-410f-a929-9914f5d3be77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:18.176266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661a5c5-4d30-410f-a929-9914f5d3be77 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:18.221020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1070/2170] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:36:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:36:19.487302 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8579df5-c566-4d70-8d2c-528950706743 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:19.489628 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f8579df5-c566-4d70-8d2c-528950706743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:19.541595 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8579df5-c566-4d70-8d2c-528950706743 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:19.555637 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbecaf-2849-4556-bd20-d9e206161e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:19.556187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbecaf-2849-4556-bd20-d9e206161e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:19.556472 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbecaf-2849-4556-bd20-d9e206161e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:19.556744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbecaf-2849-4556-bd20-d9e206161e85 admin admin] PATH_INFO: `/v3/projects/b151bad1a9184717b83bd6726d169504` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:19.562079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cbecaf-2849-4556-bd20-d9e206161e85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b151bad1a9184717b83bd6726d169504)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:19.565333 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cbecaf-2849-4556-bd20-d9e206161e85 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:19.691370 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e5cbecaf-2849-4556-bd20-d9e206161e85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b151bad1a9184717b83bd6726d169504', 'request_id': 'req-e5cbecaf-2849-4556-bd20-d9e206161e85'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:19.703178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e5cbecaf-2849-4556-bd20-d9e206161e85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b151bad1a9184717b83bd6726d169504', 'request_id': 'req-e5cbecaf-2849-4556-bd20-d9e206161e85'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:19.729318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e5cbecaf-2849-4556-bd20-d9e206161e85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b151bad1a9184717b83bd6726d169504', 'request_id': 'req-e5cbecaf-2849-4556-bd20-d9e206161e85'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:19.764069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e5cbecaf-2849-4556-bd20-d9e206161e85 admin admin] The token cache is being invalidate because project b151bad1a9184717b83bd6726d169504 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:36:19.764681 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e5cbecaf-2849-4556-bd20-d9e206161e85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:19.765692 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e5cbecaf-2849-4556-bd20-d9e206161e85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:19.772129 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1101/2171] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:36:19 2022] DELETE /identity/v3/projects/b151bad1a9184717b83bd6726d169504 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:19.776523 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661a5c5-4d30-410f-a929-9914f5d3be77 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:19.789919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661a5c5-4d30-410f-a929-9914f5d3be77 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:20.007232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794de87c-737c-49f9-b7ca-aca2966e4eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:20.008600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794de87c-737c-49f9-b7ca-aca2966e4eaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:20.008912 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794de87c-737c-49f9-b7ca-aca2966e4eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:20.009197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794de87c-737c-49f9-b7ca-aca2966e4eaf admin admin] PATH_INFO: `/v3/users/2f0ccc26b642456992521502e04966d5` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:20.010875 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794de87c-737c-49f9-b7ca-aca2966e4eaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f0ccc26b642456992521502e04966d5)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:20.015081 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794de87c-737c-49f9-b7ca-aca2966e4eaf admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:20.119168 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95b6e6-8212-4db7-a1c9-b0ef8fb8072d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:20.119320 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95b6e6-8212-4db7-a1c9-b0ef8fb8072d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:20.119589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95b6e6-8212-4db7-a1c9-b0ef8fb8072d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:20.168585 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-794de87c-737c-49f9-b7ca-aca2966e4eaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f0ccc26b642456992521502e04966d5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:20.170354 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-794de87c-737c-49f9-b7ca-aca2966e4eaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f0ccc26b642456992521502e04966d5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:20.189555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-0e95b6e6-8212-4db7-a1c9-b0ef8fb8072d None None] MFA Rules not processed for user `23e07f301bb848188defaddfa9db59b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:20.191225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0e95b6e6-8212-4db7-a1c9-b0ef8fb8072d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:20.193693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-794de87c-737c-49f9-b7ca-aca2966e4eaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f0ccc26b642456992521502e04966d5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:20.211885 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-794de87c-737c-49f9-b7ca-aca2966e4eaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f0ccc26b642456992521502e04966d5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:20.212924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-794de87c-737c-49f9-b7ca-aca2966e4eaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f0ccc26b642456992521502e04966d5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:20.241671 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1071/2172] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:36:19 2022] DELETE /identity/v3/users/2f0ccc26b642456992521502e04966d5 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:20.319275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1102/2173] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:20 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:20.335885 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794de87c-737c-49f9-b7ca-aca2966e4eaf admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:20.339261 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-794de87c-737c-49f9-b7ca-aca2966e4eaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:20.463787 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11ae4c-e526-42b6-93ff-bf8c445f7ea7 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:20.464261 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11ae4c-e526-42b6-93ff-bf8c445f7ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:20.464709 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11ae4c-e526-42b6-93ff-bf8c445f7ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:20.500278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:20.515160 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-bd11ae4c-e526-42b6-93ff-bf8c445f7ea7 None None] MFA Rules not processed for user `4001e8ec57b34f57aae1654fad9217f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:20.519316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-bd11ae4c-e526-42b6-93ff-bf8c445f7ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:20.636750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:20.637344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794de87c-737c-49f9-b7ca-aca2966e4eaf admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:20.647653 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70ea98-5906-4f5f-b3ed-04e23a350248 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:20.649503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70ea98-5906-4f5f-b3ed-04e23a350248 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:20.649870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70ea98-5906-4f5f-b3ed-04e23a350248 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:20.650535 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70ea98-5906-4f5f-b3ed-04e23a350248 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:20.674951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1b70ea98-5906-4f5f-b3ed-04e23a350248 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:20.718299 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1103/2174] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:20.803569 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70ea98-5906-4f5f-b3ed-04e23a350248 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:20.805712 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70ea98-5906-4f5f-b3ed-04e23a350248 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:20.823146 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1072/2175] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:36:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:36:21.032095 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11ae4c-e526-42b6-93ff-bf8c445f7ea7 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:21.035845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-bd11ae4c-e526-42b6-93ff-bf8c445f7ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:21.274154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11ae4c-e526-42b6-93ff-bf8c445f7ea7 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:21.304972 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de3538-9a36-40a6-b900-755cbea6bf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:21.307313 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de3538-9a36-40a6-b900-755cbea6bf4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:21.309351 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de3538-9a36-40a6-b900-755cbea6bf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:21.309586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de3538-9a36-40a6-b900-755cbea6bf4d admin admin] PATH_INFO: `/v3/projects/12b3a794f28249079677cd993fcc7618` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:21.314185 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de3538-9a36-40a6-b900-755cbea6bf4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=12b3a794f28249079677cd993fcc7618)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:21.320432 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de3538-9a36-40a6-b900-755cbea6bf4d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:21.544383 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c3de3538-9a36-40a6-b900-755cbea6bf4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12b3a794f28249079677cd993fcc7618', 'request_id': 'req-c3de3538-9a36-40a6-b900-755cbea6bf4d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:21.555375 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c3de3538-9a36-40a6-b900-755cbea6bf4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12b3a794f28249079677cd993fcc7618', 'request_id': 'req-c3de3538-9a36-40a6-b900-755cbea6bf4d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:21.580008 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c3de3538-9a36-40a6-b900-755cbea6bf4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12b3a794f28249079677cd993fcc7618', 'request_id': 'req-c3de3538-9a36-40a6-b900-755cbea6bf4d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:21.623394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c3de3538-9a36-40a6-b900-755cbea6bf4d admin admin] The token cache is being invalidate because project 12b3a794f28249079677cd993fcc7618 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:36:21.623910 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c3de3538-9a36-40a6-b900-755cbea6bf4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:21.626490 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c3de3538-9a36-40a6-b900-755cbea6bf4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:21.639577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1104/2176] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:36:21 2022] DELETE /identity/v3/projects/12b3a794f28249079677cd993fcc7618 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:21.641558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70ea98-5906-4f5f-b3ed-04e23a350248 service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:21.680448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70ea98-5906-4f5f-b3ed-04e23a350248 service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:21.884778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd83025-fb2b-465d-aa18-be28ef6ab876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:21.887167 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd83025-fb2b-465d-aa18-be28ef6ab876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:21.888825 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd83025-fb2b-465d-aa18-be28ef6ab876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:21.889169 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd83025-fb2b-465d-aa18-be28ef6ab876 admin admin] PATH_INFO: `/v3/users/00d4965e5270403d91754e47e58d9eb7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:21.893208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd83025-fb2b-465d-aa18-be28ef6ab876 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00d4965e5270403d91754e47e58d9eb7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:21.899549 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd83025-fb2b-465d-aa18-be28ef6ab876 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:22.092113 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cfd83025-fb2b-465d-aa18-be28ef6ab876 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00d4965e5270403d91754e47e58d9eb7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:22.104363 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cfd83025-fb2b-465d-aa18-be28ef6ab876 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00d4965e5270403d91754e47e58d9eb7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:22.129067 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cfd83025-fb2b-465d-aa18-be28ef6ab876 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00d4965e5270403d91754e47e58d9eb7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:22.159800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cfd83025-fb2b-465d-aa18-be28ef6ab876 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00d4965e5270403d91754e47e58d9eb7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:22.163546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cfd83025-fb2b-465d-aa18-be28ef6ab876 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00d4965e5270403d91754e47e58d9eb7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:22.166509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:22.204022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1073/2177] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:36:21 2022] DELETE /identity/v3/users/00d4965e5270403d91754e47e58d9eb7 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:22.880151 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de3538-9a36-40a6-b900-755cbea6bf4d admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:22.908090 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c3de3538-9a36-40a6-b900-755cbea6bf4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:22.953979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de3538-9a36-40a6-b900-755cbea6bf4d admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:23.196187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391fb503-6418-4612-a79a-8ac8cb90af00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:23.199836 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fb503-6418-4612-a79a-8ac8cb90af00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:23.200951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fb503-6418-4612-a79a-8ac8cb90af00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:23.201864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fb503-6418-4612-a79a-8ac8cb90af00 admin admin] PATH_INFO: `/v3/projects/d919a8c5075a483fa8a73d5ff54e0746` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:23.205354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391fb503-6418-4612-a79a-8ac8cb90af00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d919a8c5075a483fa8a73d5ff54e0746)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:23.215730 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391fb503-6418-4612-a79a-8ac8cb90af00 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:23.265793 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:23.363537 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-391fb503-6418-4612-a79a-8ac8cb90af00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd919a8c5075a483fa8a73d5ff54e0746', 'request_id': 'req-391fb503-6418-4612-a79a-8ac8cb90af00'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:23.376285 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-391fb503-6418-4612-a79a-8ac8cb90af00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd919a8c5075a483fa8a73d5ff54e0746', 'request_id': 'req-391fb503-6418-4612-a79a-8ac8cb90af00'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:23.394407 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-391fb503-6418-4612-a79a-8ac8cb90af00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd919a8c5075a483fa8a73d5ff54e0746', 'request_id': 'req-391fb503-6418-4612-a79a-8ac8cb90af00'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:23.428862 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-391fb503-6418-4612-a79a-8ac8cb90af00 admin admin] The token cache is being invalidate because project d919a8c5075a483fa8a73d5ff54e0746 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:36:23.429339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-391fb503-6418-4612-a79a-8ac8cb90af00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:23.432114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-391fb503-6418-4612-a79a-8ac8cb90af00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:23.435723 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1105/2178] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:36:22 2022] DELETE /identity/v3/projects/d919a8c5075a483fa8a73d5ff54e0746 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:23.440755 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f1cab-e1c6-4361-9145-56bbfb959016 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:23.441444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f1cab-e1c6-4361-9145-56bbfb959016 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:23.441788 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f1cab-e1c6-4361-9145-56bbfb959016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:23.497766 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-bd4f1cab-e1c6-4361-9145-56bbfb959016 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:23.500756 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bd4f1cab-e1c6-4361-9145-56bbfb959016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:23.623849 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1074/2179] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:23.632520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ce6bf-c5c2-4071-9f73-cb5075bfa896 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:23.633412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ce6bf-c5c2-4071-9f73-cb5075bfa896 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:23.633875 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ce6bf-c5c2-4071-9f73-cb5075bfa896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:23.697028 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-bf3ce6bf-c5c2-4071-9f73-cb5075bfa896 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:23.703183 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-bf3ce6bf-c5c2-4071-9f73-cb5075bfa896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:23.717906 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1106/2180] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:23.727742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f1cab-e1c6-4361-9145-56bbfb959016 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:23.761990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f1cab-e1c6-4361-9145-56bbfb959016 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:23.774956 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba4b7b-23f0-4465-9eae-7f2c987d3be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:23.776238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba4b7b-23f0-4465-9eae-7f2c987d3be1 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:23.777022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba4b7b-23f0-4465-9eae-7f2c987d3be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:23.778207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba4b7b-23f0-4465-9eae-7f2c987d3be1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:23.780762 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba4b7b-23f0-4465-9eae-7f2c987d3be1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:36:23.781277 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba4b7b-23f0-4465-9eae-7f2c987d3be1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:23.783002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba4b7b-23f0-4465-9eae-7f2c987d3be1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:23.830313 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ce6bf-c5c2-4071-9f73-cb5075bfa896 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:23.830797 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1075/2181] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:36:23.846194 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ce6bf-c5c2-4071-9f73-cb5075bfa896 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:23.855343 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa91fa-50cb-46dc-9c62-09e3f5dce588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:23.856554 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa91fa-50cb-46dc-9c62-09e3f5dce588 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:23.856834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa91fa-50cb-46dc-9c62-09e3f5dce588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:23.857109 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa91fa-50cb-46dc-9c62-09e3f5dce588 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:23.861930 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa91fa-50cb-46dc-9c62-09e3f5dce588 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:23.864472 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa91fa-50cb-46dc-9c62-09e3f5dce588 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:23.933383 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1107/2182] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:36:23 2022] POST /identity/v3/projects => generated 399 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:36:23.938562 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba4b7b-23f0-4465-9eae-7f2c987d3be1 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:23.954426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba4b7b-23f0-4465-9eae-7f2c987d3be1 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:24.123547 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48553689-2cde-4a37-8611-d2bbbafc2a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:24.124602 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48553689-2cde-4a37-8611-d2bbbafc2a02 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:24.125094 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48553689-2cde-4a37-8611-d2bbbafc2a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:24.125583 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48553689-2cde-4a37-8611-d2bbbafc2a02 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:24.128174 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48553689-2cde-4a37-8611-d2bbbafc2a02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:24.131816 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48553689-2cde-4a37-8611-d2bbbafc2a02 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:24.182429 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1076/2183] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:36:23 2022] POST /identity/v3/users => generated 300 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:36:24.186589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa91fa-50cb-46dc-9c62-09e3f5dce588 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:24.226300 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa91fa-50cb-46dc-9c62-09e3f5dce588 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:24.259510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c4ef9-2fad-4ae2-92d7-671f9bc892b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:24.260264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c4ef9-2fad-4ae2-92d7-671f9bc892b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:24.260572 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c4ef9-2fad-4ae2-92d7-671f9bc892b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:24.270553 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c4ef9-2fad-4ae2-92d7-671f9bc892b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:24.274251 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c4ef9-2fad-4ae2-92d7-671f9bc892b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:24.279906 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c4ef9-2fad-4ae2-92d7-671f9bc892b7 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:24.311813 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1108/2184] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:36:24.313112 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48553689-2cde-4a37-8611-d2bbbafc2a02 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:24.329299 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48553689-2cde-4a37-8611-d2bbbafc2a02 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:24.348222 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f14d1-d141-41c5-997f-5182f233f7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:24.348709 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f14d1-d141-41c5-997f-5182f233f7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:24.349153 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f14d1-d141-41c5-997f-5182f233f7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:24.349445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f14d1-d141-41c5-997f-5182f233f7a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:24.350426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f14d1-d141-41c5-997f-5182f233f7a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:24.354951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f14d1-d141-41c5-997f-5182f233f7a2 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:24.388688 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1077/2185] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:36:24.394272 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c4ef9-2fad-4ae2-92d7-671f9bc892b7 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:24.407217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c4ef9-2fad-4ae2-92d7-671f9bc892b7 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:24.420862 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e91b51-5dcc-4209-a823-317327f4c7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:24.422744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e91b51-5dcc-4209-a823-317327f4c7cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:24.424958 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e91b51-5dcc-4209-a823-317327f4c7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:24.425540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e91b51-5dcc-4209-a823-317327f4c7cf admin admin] PATH_INFO: `/v3/projects/2dd20d4f8cbb42efb11ef52387a804e5/users/98ec62044ba84609a5adb6794faad774/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:24.468987 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e91b51-5dcc-4209-a823-317327f4c7cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd20d4f8cbb42efb11ef52387a804e5, user_id=98ec62044ba84609a5adb6794faad774, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:24.473368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e91b51-5dcc-4209-a823-317327f4c7cf admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:24.512476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1109/2186] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:36:24 2022] PUT /identity/v3/projects/2dd20d4f8cbb42efb11ef52387a804e5/users/98ec62044ba84609a5adb6794faad774/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:24.529385 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66f6ac-8308-45a8-bc8c-8b00420feb78 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:24.530720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66f6ac-8308-45a8-bc8c-8b00420feb78 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:24.531839 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66f6ac-8308-45a8-bc8c-8b00420feb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:24.593273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-bb66f6ac-8308-45a8-bc8c-8b00420feb78 None None] MFA Rules not processed for user `98ec62044ba84609a5adb6794faad774`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:24.597114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bb66f6ac-8308-45a8-bc8c-8b00420feb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:24.746800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:24.859493 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1078/2187] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:24.887156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e91b51-5dcc-4209-a823-317327f4c7cf admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:24.891140 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-13e91b51-5dcc-4209-a823-317327f4c7cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:24.943626 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b77d1-3a3e-44f3-a072-17d5cfd8b0ed None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:24.944295 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b77d1-3a3e-44f3-a072-17d5cfd8b0ed None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:24.944905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b77d1-3a3e-44f3-a072-17d5cfd8b0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:25.050516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-607b77d1-3a3e-44f3-a072-17d5cfd8b0ed None None] MFA Rules not processed for user `6d903b4429664fceaa1344f1b99791dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:25.053849 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-607b77d1-3a3e-44f3-a072-17d5cfd8b0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:25.080467 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e91b51-5dcc-4209-a823-317327f4c7cf admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:25.114764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63be624-2e80-4522-b5dc-094fcbe42a64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:25.115486 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63be624-2e80-4522-b5dc-094fcbe42a64 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:25.115933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63be624-2e80-4522-b5dc-094fcbe42a64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:25.116384 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63be624-2e80-4522-b5dc-094fcbe42a64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:25.140771 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63be624-2e80-4522-b5dc-094fcbe42a64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:25.142602 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63be624-2e80-4522-b5dc-094fcbe42a64 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:25.173078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1110/2188] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:36:24 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:36:25.325317 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1079/2189] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:35.154956 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:35.290566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=31 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:36.867752 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991ad1a-d8d4-43fa-bfbe-73319513fc73 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:36.868279 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991ad1a-d8d4-43fa-bfbe-73319513fc73 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:36.868279 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991ad1a-d8d4-43fa-bfbe-73319513fc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:36.890292 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1991ad1a-d8d4-43fa-bfbe-73319513fc73 None None] MFA Rules not processed for user `f36c6987ef774a1684c55f885e2a4f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:36.891546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1991ad1a-d8d4-43fa-bfbe-73319513fc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:36.983010 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1111/2190] 10.222.0.83 () {58 vars in 967 bytes} [Mon Aug 1 15:36:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:36.992316 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b77d1-3a3e-44f3-a072-17d5cfd8b0ed None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:36.994155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-607b77d1-3a3e-44f3-a072-17d5cfd8b0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:37.134762 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b77d1-3a3e-44f3-a072-17d5cfd8b0ed None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:37.144687 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6287d5-089a-47e5-b2f0-eb266071faec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6873a3e440bd4b63b0082fc85beef49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:37.145617 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6287d5-089a-47e5-b2f0-eb266071faec service swift] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:37.146149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6287d5-089a-47e5-b2f0-eb266071faec service swift] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:37.146604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6287d5-089a-47e5-b2f0-eb266071faec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:37.178935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6287d5-089a-47e5-b2f0-eb266071faec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:37.180630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6287d5-089a-47e5-b2f0-eb266071faec service swift] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:37.191526 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1080/2191] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Aug 1 15:36:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:36:37.788783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991ad1a-d8d4-43fa-bfbe-73319513fc73 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:37.869626 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991ad1a-d8d4-43fa-bfbe-73319513fc73 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:37.874974 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56d224-e8ff-4acc-a517-c9208f9616cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:37.875644 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56d224-e8ff-4acc-a517-c9208f9616cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:37.875924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56d224-e8ff-4acc-a517-c9208f9616cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:37.876200 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56d224-e8ff-4acc-a517-c9208f9616cc admin admin] PATH_INFO: `/v3/users/98ec62044ba84609a5adb6794faad774` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:37.877584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec56d224-e8ff-4acc-a517-c9208f9616cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ec62044ba84609a5adb6794faad774)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:37.879362 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec56d224-e8ff-4acc-a517-c9208f9616cc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:37.939405 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ec56d224-e8ff-4acc-a517-c9208f9616cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ec62044ba84609a5adb6794faad774'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:37.940153 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ec56d224-e8ff-4acc-a517-c9208f9616cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ec62044ba84609a5adb6794faad774'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:37.954361 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ec56d224-e8ff-4acc-a517-c9208f9616cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ec62044ba84609a5adb6794faad774'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:37.972314 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ec56d224-e8ff-4acc-a517-c9208f9616cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ec62044ba84609a5adb6794faad774'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:37.981488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ec56d224-e8ff-4acc-a517-c9208f9616cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ec62044ba84609a5adb6794faad774'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:37.988383 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1112/2192] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:36:37 2022] DELETE /identity/v3/users/98ec62044ba84609a5adb6794faad774 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:37.995200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7e075-fbf8-4fe3-b9b4-b6c90baf3e1f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:37.996268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7e075-fbf8-4fe3-b9b4-b6c90baf3e1f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:37.996268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7e075-fbf8-4fe3-b9b4-b6c90baf3e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:38.024404 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4fe7e075-fbf8-4fe3-b9b4-b6c90baf3e1f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:38.026203 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4fe7e075-fbf8-4fe3-b9b4-b6c90baf3e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:38.099448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1081/2193] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:36:38.109990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56d224-e8ff-4acc-a517-c9208f9616cc admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:38.111446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ec56d224-e8ff-4acc-a517-c9208f9616cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:38.141667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:38.177730 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56d224-e8ff-4acc-a517-c9208f9616cc admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:38.181488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1b425-61fe-4046-a9a8-8e1e40417693 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:38.182135 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1b425-61fe-4046-a9a8-8e1e40417693 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:38.182443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1b425-61fe-4046-a9a8-8e1e40417693 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:38.183097 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1b425-61fe-4046-a9a8-8e1e40417693 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:38.202011 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1b425-61fe-4046-a9a8-8e1e40417693 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:38.203472 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1b425-61fe-4046-a9a8-8e1e40417693 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:38.214397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1113/2194] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:36:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:36:38.596472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7e075-fbf8-4fe3-b9b4-b6c90baf3e1f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:38.596472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4fe7e075-fbf8-4fe3-b9b4-b6c90baf3e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:38.611464 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7e075-fbf8-4fe3-b9b4-b6c90baf3e1f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:38.617212 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615f081-4e89-418f-b0b2-6f2c84654a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:38.618184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615f081-4e89-418f-b0b2-6f2c84654a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:38.618399 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615f081-4e89-418f-b0b2-6f2c84654a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:38.618736 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615f081-4e89-418f-b0b2-6f2c84654a99 admin admin] PATH_INFO: `/v3/projects/2dd20d4f8cbb42efb11ef52387a804e5` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:38.619970 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615f081-4e89-418f-b0b2-6f2c84654a99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dd20d4f8cbb42efb11ef52387a804e5)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:38.622003 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615f081-4e89-418f-b0b2-6f2c84654a99 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:38.690925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3615f081-4e89-418f-b0b2-6f2c84654a99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dd20d4f8cbb42efb11ef52387a804e5', 'request_id': 'req-3615f081-4e89-418f-b0b2-6f2c84654a99'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:38.698070 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3615f081-4e89-418f-b0b2-6f2c84654a99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dd20d4f8cbb42efb11ef52387a804e5', 'request_id': 'req-3615f081-4e89-418f-b0b2-6f2c84654a99'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:38.704458 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3615f081-4e89-418f-b0b2-6f2c84654a99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dd20d4f8cbb42efb11ef52387a804e5', 'request_id': 'req-3615f081-4e89-418f-b0b2-6f2c84654a99'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:38.720347 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3615f081-4e89-418f-b0b2-6f2c84654a99 admin admin] The token cache is being invalidate because project 2dd20d4f8cbb42efb11ef52387a804e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:36:38.720641 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3615f081-4e89-418f-b0b2-6f2c84654a99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:38.721323 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3615f081-4e89-418f-b0b2-6f2c84654a99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:38.722918 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1082/2195] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:36:38 2022] DELETE /identity/v3/projects/2dd20d4f8cbb42efb11ef52387a804e5 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:38.729055 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847facc-4410-465e-b9da-c86bf40d0897 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:38.729425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847facc-4410-465e-b9da-c86bf40d0897 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:38.729708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847facc-4410-465e-b9da-c86bf40d0897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:38.751312 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e847facc-4410-465e-b9da-c86bf40d0897 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:38.752556 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e847facc-4410-465e-b9da-c86bf40d0897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:38.832427 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1114/2196] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:36:38.834229 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d781362-64e4-4739-a838-08cebd1cea50 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:38.834626 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d781362-64e4-4739-a838-08cebd1cea50 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:38.834909 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d781362-64e4-4739-a838-08cebd1cea50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:38.863807 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4d781362-64e4-4739-a838-08cebd1cea50 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:38.865113 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4d781362-64e4-4739-a838-08cebd1cea50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:38.880918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847facc-4410-465e-b9da-c86bf40d0897 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:38.882119 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1083/2197] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:38.892995 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847facc-4410-465e-b9da-c86bf40d0897 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:38.899971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca564c-23fe-46c4-85dc-4dd1e31c2f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:38.914182 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca564c-23fe-46c4-85dc-4dd1e31c2f6c admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:38.914182 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca564c-23fe-46c4-85dc-4dd1e31c2f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:38.914182 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca564c-23fe-46c4-85dc-4dd1e31c2f6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:38.914182 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca564c-23fe-46c4-85dc-4dd1e31c2f6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:36:38.914182 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca564c-23fe-46c4-85dc-4dd1e31c2f6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:38.914182 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca564c-23fe-46c4-85dc-4dd1e31c2f6c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:38.921348 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1115/2198] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:36:38.926951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d781362-64e4-4739-a838-08cebd1cea50 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:38.936998 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d781362-64e4-4739-a838-08cebd1cea50 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:38.942041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5960c9f-573e-42e3-bf8b-6c01a96cf69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:38.942671 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5960c9f-573e-42e3-bf8b-6c01a96cf69e admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:38.942947 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5960c9f-573e-42e3-bf8b-6c01a96cf69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:38.943219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5960c9f-573e-42e3-bf8b-6c01a96cf69e admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:38.943889 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5960c9f-573e-42e3-bf8b-6c01a96cf69e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:38.945808 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5960c9f-573e-42e3-bf8b-6c01a96cf69e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:38.951237 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:38.986012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1084/2199] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:36:38 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:36:38.989134 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca564c-23fe-46c4-85dc-4dd1e31c2f6c admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:39.000362 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca564c-23fe-46c4-85dc-4dd1e31c2f6c admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:39.090760 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7026f60b-68ba-4a2b-b68f-4f89038660fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:39.091329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026f60b-68ba-4a2b-b68f-4f89038660fa admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:39.091833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026f60b-68ba-4a2b-b68f-4f89038660fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:39.092063 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026f60b-68ba-4a2b-b68f-4f89038660fa admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:39.093235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7026f60b-68ba-4a2b-b68f-4f89038660fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:39.094770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7026f60b-68ba-4a2b-b68f-4f89038660fa admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:39.124454 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1116/2200] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:36:38 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:36:39.133565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5960c9f-573e-42e3-bf8b-6c01a96cf69e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:39.147861 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5960c9f-573e-42e3-bf8b-6c01a96cf69e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:39.153870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641944e2-f0f9-464b-8117-fcdfce1abdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:39.154535 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641944e2-f0f9-464b-8117-fcdfce1abdc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:39.154824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641944e2-f0f9-464b-8117-fcdfce1abdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:39.155086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641944e2-f0f9-464b-8117-fcdfce1abdc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:39.155792 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641944e2-f0f9-464b-8117-fcdfce1abdc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:39.157416 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641944e2-f0f9-464b-8117-fcdfce1abdc9 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:39.168490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1085/2201] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:36:39.172757 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7026f60b-68ba-4a2b-b68f-4f89038660fa admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:39.183102 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7026f60b-68ba-4a2b-b68f-4f89038660fa admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:39.187947 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9c134-ef31-4425-ab2d-ba21ae35846b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:39.188576 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9c134-ef31-4425-ab2d-ba21ae35846b admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:39.188852 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9c134-ef31-4425-ab2d-ba21ae35846b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:39.189123 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9c134-ef31-4425-ab2d-ba21ae35846b admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:39.189851 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9c134-ef31-4425-ab2d-ba21ae35846b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:39.191568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9c134-ef31-4425-ab2d-ba21ae35846b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:39.203413 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1117/2202] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:36:39.207881 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641944e2-f0f9-464b-8117-fcdfce1abdc9 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:39.222004 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641944e2-f0f9-464b-8117-fcdfce1abdc9 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:39.230083 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea5f5b-95cd-45a6-b114-b102d3628cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:39.231194 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea5f5b-95cd-45a6-b114-b102d3628cb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:39.231665 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea5f5b-95cd-45a6-b114-b102d3628cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:39.232119 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea5f5b-95cd-45a6-b114-b102d3628cb0 admin admin] PATH_INFO: `/v3/projects/3762799522514a268a4c8b8069baa882/users/a26637889378427eb884eed624606fd5/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:39.250856 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea5f5b-95cd-45a6-b114-b102d3628cb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3762799522514a268a4c8b8069baa882, user_id=a26637889378427eb884eed624606fd5, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:39.253514 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea5f5b-95cd-45a6-b114-b102d3628cb0 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:39.267349 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1086/2203] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:36:39 2022] PUT /identity/v3/projects/3762799522514a268a4c8b8069baa882/users/a26637889378427eb884eed624606fd5/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:36:39.272370 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b94c1-df4a-475a-997f-f6c0bdb455f4 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:39.272660 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b94c1-df4a-475a-997f-f6c0bdb455f4 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:39.272930 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b94c1-df4a-475a-997f-f6c0bdb455f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:39.293205 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-017b94c1-df4a-475a-997f-f6c0bdb455f4 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:39.295158 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-017b94c1-df4a-475a-997f-f6c0bdb455f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:39.329788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:39.366708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1118/2204] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:39.377980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea5f5b-95cd-45a6-b114-b102d3628cb0 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:39.379693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7aea5f5b-95cd-45a6-b114-b102d3628cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:39.475338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea5f5b-95cd-45a6-b114-b102d3628cb0 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:39.479041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81007f14-aeaa-4946-92a6-c8597208da0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:39.479672 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81007f14-aeaa-4946-92a6-c8597208da0b service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:39.479974 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81007f14-aeaa-4946-92a6-c8597208da0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:39.480411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81007f14-aeaa-4946-92a6-c8597208da0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:39.490846 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81007f14-aeaa-4946-92a6-c8597208da0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:39.492318 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81007f14-aeaa-4946-92a6-c8597208da0b service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:39.503371 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1087/2205] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:36:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:36:42.357716 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ffa51-0394-40fa-aee8-0a508c689d61 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:42.358256 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ffa51-0394-40fa-aee8-0a508c689d61 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:42.358340 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ffa51-0394-40fa-aee8-0a508c689d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:42.402246 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-b05ffa51-0394-40fa-aee8-0a508c689d61 None None] MFA Rules not processed for user `f36c6987ef774a1684c55f885e2a4f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:42.403159 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b05ffa51-0394-40fa-aee8-0a508c689d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:42.505948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1119/2206] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Aug 1 15:36:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:42.525112 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81007f14-aeaa-4946-92a6-c8597208da0b service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:42.530291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-81007f14-aeaa-4946-92a6-c8597208da0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:42.757727 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81007f14-aeaa-4946-92a6-c8597208da0b service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:42.781950 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f48cf-d8fd-4b34-8bc0-a766be627042 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6873a3e440bd4b63b0082fc85beef49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:42.782658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f48cf-d8fd-4b34-8bc0-a766be627042 service swift] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:42.782962 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f48cf-d8fd-4b34-8bc0-a766be627042 service swift] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:42.783246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f48cf-d8fd-4b34-8bc0-a766be627042 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:42.811714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f48cf-d8fd-4b34-8bc0-a766be627042 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:42.816785 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f48cf-d8fd-4b34-8bc0-a766be627042 service swift] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:42.859091 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1088/2207] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Aug 1 15:36:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:36:43.973266 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ffa51-0394-40fa-aee8-0a508c689d61 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:43.981772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b05ffa51-0394-40fa-aee8-0a508c689d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:44.021998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ffa51-0394-40fa-aee8-0a508c689d61 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:44.037733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9344219-b4b7-4849-8a91-84052f202836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:44.039427 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9344219-b4b7-4849-8a91-84052f202836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:44.040279 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9344219-b4b7-4849-8a91-84052f202836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:44.040776 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9344219-b4b7-4849-8a91-84052f202836 admin admin] PATH_INFO: `/v3/users/c6591fd1ff63406c86b847870570e56c` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:44.048327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9344219-b4b7-4849-8a91-84052f202836 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6591fd1ff63406c86b847870570e56c)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:44.051936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9344219-b4b7-4849-8a91-84052f202836 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:44.145133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d9344219-b4b7-4849-8a91-84052f202836 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6591fd1ff63406c86b847870570e56c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:44.147076 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d9344219-b4b7-4849-8a91-84052f202836 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6591fd1ff63406c86b847870570e56c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:44.162638 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d9344219-b4b7-4849-8a91-84052f202836 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6591fd1ff63406c86b847870570e56c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:44.228784 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d9344219-b4b7-4849-8a91-84052f202836 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6591fd1ff63406c86b847870570e56c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:44.246791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d9344219-b4b7-4849-8a91-84052f202836 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6591fd1ff63406c86b847870570e56c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:44.279357 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1120/2208] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:36:43 2022] DELETE /identity/v3/users/c6591fd1ff63406c86b847870570e56c => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:44.665697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f48cf-d8fd-4b34-8bc0-a766be627042 service swift] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:44.700501 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f48cf-d8fd-4b34-8bc0-a766be627042 service swift] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:44.854729 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:44.901590 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79561827-d2b3-4e93-a5af-78496e8fa066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:44.903424 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79561827-d2b3-4e93-a5af-78496e8fa066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:44.904037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79561827-d2b3-4e93-a5af-78496e8fa066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:44.904920 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79561827-d2b3-4e93-a5af-78496e8fa066 admin admin] PATH_INFO: `/v3/projects/8454f4c13e654ee58e6ff6265b7b85f9` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:44.907807 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79561827-d2b3-4e93-a5af-78496e8fa066 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8454f4c13e654ee58e6ff6265b7b85f9)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:44.913658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79561827-d2b3-4e93-a5af-78496e8fa066 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:45.013095 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-79561827-d2b3-4e93-a5af-78496e8fa066 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8454f4c13e654ee58e6ff6265b7b85f9', 'request_id': 'req-79561827-d2b3-4e93-a5af-78496e8fa066'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:45.023515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-79561827-d2b3-4e93-a5af-78496e8fa066 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8454f4c13e654ee58e6ff6265b7b85f9', 'request_id': 'req-79561827-d2b3-4e93-a5af-78496e8fa066'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:45.035031 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-79561827-d2b3-4e93-a5af-78496e8fa066 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8454f4c13e654ee58e6ff6265b7b85f9', 'request_id': 'req-79561827-d2b3-4e93-a5af-78496e8fa066'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:45.088076 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-79561827-d2b3-4e93-a5af-78496e8fa066 admin admin] The token cache is being invalidate because project 8454f4c13e654ee58e6ff6265b7b85f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:36:45.088704 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-79561827-d2b3-4e93-a5af-78496e8fa066 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:45.099442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-79561827-d2b3-4e93-a5af-78496e8fa066 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:45.101020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1089/2209] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:36:44 2022] DELETE /identity/v3/projects/8454f4c13e654ee58e6ff6265b7b85f9 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:36:45.105602 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109e739-5f73-4774-8b96-e118c8f5387c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:45.106186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109e739-5f73-4774-8b96-e118c8f5387c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:45.106186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109e739-5f73-4774-8b96-e118c8f5387c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:45.175799 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-b109e739-5f73-4774-8b96-e118c8f5387c None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:45.177475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b109e739-5f73-4774-8b96-e118c8f5387c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:45.335129 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:45.383808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1121/2210] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:45.392899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba889070-4cd3-4079-b2d5-f270b6265338 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:45.393649 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba889070-4cd3-4079-b2d5-f270b6265338 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:45.394337 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba889070-4cd3-4079-b2d5-f270b6265338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:45.449137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ba889070-4cd3-4079-b2d5-f270b6265338 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:45.451103 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ba889070-4cd3-4079-b2d5-f270b6265338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:45.464003 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1090/2211] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:45.468961 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109e739-5f73-4774-8b96-e118c8f5387c None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:45.491325 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109e739-5f73-4774-8b96-e118c8f5387c None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:45.510836 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e011fa7-d6a6-457d-bb3c-290d214786fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:45.511546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e011fa7-d6a6-457d-bb3c-290d214786fe admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:45.511846 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e011fa7-d6a6-457d-bb3c-290d214786fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:45.512122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e011fa7-d6a6-457d-bb3c-290d214786fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:45.512844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e011fa7-d6a6-457d-bb3c-290d214786fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:36:45.513153 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e011fa7-d6a6-457d-bb3c-290d214786fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:45.517391 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e011fa7-d6a6-457d-bb3c-290d214786fe admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:45.567400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1122/2212] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:36:45.569324 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba889070-4cd3-4079-b2d5-f270b6265338 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:45.585370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba889070-4cd3-4079-b2d5-f270b6265338 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:45.598489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e77869-05a5-40ba-adad-224b73eb6b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:45.599331 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e77869-05a5-40ba-adad-224b73eb6b29 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:45.599805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e77869-05a5-40ba-adad-224b73eb6b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:45.600194 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e77869-05a5-40ba-adad-224b73eb6b29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:45.601194 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e77869-05a5-40ba-adad-224b73eb6b29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:45.603486 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e77869-05a5-40ba-adad-224b73eb6b29 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:45.682873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1091/2213] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:36:45 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:36:45.689941 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e011fa7-d6a6-457d-bb3c-290d214786fe admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:45.712328 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e011fa7-d6a6-457d-bb3c-290d214786fe admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:45.880814 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f52282-633e-4f77-9873-4cec6bcce355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:45.881540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f52282-633e-4f77-9873-4cec6bcce355 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:45.881821 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f52282-633e-4f77-9873-4cec6bcce355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:45.882690 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f52282-633e-4f77-9873-4cec6bcce355 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:45.883403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f52282-633e-4f77-9873-4cec6bcce355 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:45.886041 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f52282-633e-4f77-9873-4cec6bcce355 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:45.920453 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1123/2214] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:36:45 2022] POST /identity/v3/users => generated 292 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:36:45.928920 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e77869-05a5-40ba-adad-224b73eb6b29 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:45.968170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e77869-05a5-40ba-adad-224b73eb6b29 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:45.995298 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90d412-fcd0-4a67-8739-1a943620ca4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:45.997095 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90d412-fcd0-4a67-8739-1a943620ca4d admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:46.004163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90d412-fcd0-4a67-8739-1a943620ca4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:46.004461 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90d412-fcd0-4a67-8739-1a943620ca4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:46.005475 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90d412-fcd0-4a67-8739-1a943620ca4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:46.015979 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90d412-fcd0-4a67-8739-1a943620ca4d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:46.077925 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f52282-633e-4f77-9873-4cec6bcce355 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:46.090420 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1092/2215] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:36:46.096232 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f52282-633e-4f77-9873-4cec6bcce355 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:46.106877 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c209350-7890-4ce2-9511-3dcf3834f358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:46.113026 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c209350-7890-4ce2-9511-3dcf3834f358 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:46.113557 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c209350-7890-4ce2-9511-3dcf3834f358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:46.115690 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c209350-7890-4ce2-9511-3dcf3834f358 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:46.116738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c209350-7890-4ce2-9511-3dcf3834f358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:46.118781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c209350-7890-4ce2-9511-3dcf3834f358 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:46.177990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1124/2216] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:36:46.182270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90d412-fcd0-4a67-8739-1a943620ca4d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:46.204987 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90d412-fcd0-4a67-8739-1a943620ca4d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:46.217467 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3685108-ccd6-4ecc-98a5-4ab9fd150eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:46.220493 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3685108-ccd6-4ecc-98a5-4ab9fd150eed admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:46.220751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3685108-ccd6-4ecc-98a5-4ab9fd150eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:46.222553 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3685108-ccd6-4ecc-98a5-4ab9fd150eed admin admin] PATH_INFO: `/v3/projects/2d1dff1a858a43fa878dfbfe7a6a8ce1/users/ec70a884d9dd44118a78c90c2b90b421/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:46.271677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3685108-ccd6-4ecc-98a5-4ab9fd150eed admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d1dff1a858a43fa878dfbfe7a6a8ce1, user_id=ec70a884d9dd44118a78c90c2b90b421, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:46.284806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3685108-ccd6-4ecc-98a5-4ab9fd150eed admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:46.317819 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1093/2217] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:36:46 2022] PUT /identity/v3/projects/2d1dff1a858a43fa878dfbfe7a6a8ce1/users/ec70a884d9dd44118a78c90c2b90b421/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:36:46.329743 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774ad90-fcf1-471a-9702-5cc57a78dcb2 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:46.330355 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774ad90-fcf1-471a-9702-5cc57a78dcb2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:46.330698 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774ad90-fcf1-471a-9702-5cc57a78dcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:46.412645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1774ad90-fcf1-471a-9702-5cc57a78dcb2 None None] MFA Rules not processed for user `ec70a884d9dd44118a78c90c2b90b421`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:46.423156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1774ad90-fcf1-471a-9702-5cc57a78dcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:46.575360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1125/2218] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:46.582331 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3685108-ccd6-4ecc-98a5-4ab9fd150eed admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:46.584153 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e3685108-ccd6-4ecc-98a5-4ab9fd150eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:46.784325 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3685108-ccd6-4ecc-98a5-4ab9fd150eed admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:46.790080 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a628bf-c2a9-4806-8c5a-e4016047a402 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:46.790929 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a628bf-c2a9-4806-8c5a-e4016047a402 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:46.791322 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a628bf-c2a9-4806-8c5a-e4016047a402 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:46.791867 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a628bf-c2a9-4806-8c5a-e4016047a402 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:46.806044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a628bf-c2a9-4806-8c5a-e4016047a402 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:46.807822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a628bf-c2a9-4806-8c5a-e4016047a402 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:46.820782 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1094/2219] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:36:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:36:47.076850 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774ad90-fcf1-471a-9702-5cc57a78dcb2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:47.084462 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1774ad90-fcf1-471a-9702-5cc57a78dcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:47.228479 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:47.269300 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774ad90-fcf1-471a-9702-5cc57a78dcb2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:47.283077 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3db57-bd11-4d82-8964-34db01f5371a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:47.284128 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3db57-bd11-4d82-8964-34db01f5371a service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:47.284693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3db57-bd11-4d82-8964-34db01f5371a service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:47.285219 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3db57-bd11-4d82-8964-34db01f5371a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:47.287591 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d6c3db57-bd11-4d82-8964-34db01f5371a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:47.308708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3db57-bd11-4d82-8964-34db01f5371a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:47.310761 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3db57-bd11-4d82-8964-34db01f5371a service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:47.364325 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1126/2220] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:36:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:36:49.828869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff73864-6e2c-4d89-9064-975037f733a7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:49.829728 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff73864-6e2c-4d89-9064-975037f733a7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:49.830150 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff73864-6e2c-4d89-9064-975037f733a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:49.886701 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:49.904515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-2ff73864-6e2c-4d89-9064-975037f733a7 None None] MFA Rules not processed for user `a26637889378427eb884eed624606fd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:49.907400 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2ff73864-6e2c-4d89-9064-975037f733a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:50.184960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1095/2221] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:36:50.189599 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3db57-bd11-4d82-8964-34db01f5371a service placement] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:50.191110 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d6c3db57-bd11-4d82-8964-34db01f5371a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:50.347970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3db57-bd11-4d82-8964-34db01f5371a service placement] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:50.395068 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5558eb-2339-48fe-9124-9be4d6063668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:50.395815 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5558eb-2339-48fe-9124-9be4d6063668 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:50.396096 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5558eb-2339-48fe-9124-9be4d6063668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:50.398943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5558eb-2339-48fe-9124-9be4d6063668 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:50.399927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5558eb-2339-48fe-9124-9be4d6063668 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:50.409052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5558eb-2339-48fe-9124-9be4d6063668 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:50.515586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1127/2222] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:36:50 2022] POST /identity/v3/projects => generated 381 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:36:50.516819 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff73864-6e2c-4d89-9064-975037f733a7 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:50.539364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff73864-6e2c-4d89-9064-975037f733a7 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:50.766390 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dba896-280a-40a3-b8f3-8613d5750ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:50.767175 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dba896-280a-40a3-b8f3-8613d5750ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:50.767496 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dba896-280a-40a3-b8f3-8613d5750ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:50.767777 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dba896-280a-40a3-b8f3-8613d5750ef4 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:50.768645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dba896-280a-40a3-b8f3-8613d5750ef4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:50.774247 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dba896-280a-40a3-b8f3-8613d5750ef4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:50.820154 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1096/2223] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:36:50 2022] POST /identity/v3/users => generated 297 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:36:50.822353 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5558eb-2339-48fe-9124-9be4d6063668 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:50.843946 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5558eb-2339-48fe-9124-9be4d6063668 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:50.862817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5b9db-0374-4fd1-a534-c1b0ef6c9365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:50.863638 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5b9db-0374-4fd1-a534-c1b0ef6c9365 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:50.864024 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5b9db-0374-4fd1-a534-c1b0ef6c9365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:50.864396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5b9db-0374-4fd1-a534-c1b0ef6c9365 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:50.865338 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5b9db-0374-4fd1-a534-c1b0ef6c9365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:50.867528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5b9db-0374-4fd1-a534-c1b0ef6c9365 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:50.902869 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1128/2224] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:36:50.905294 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dba896-280a-40a3-b8f3-8613d5750ef4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:50.931013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dba896-280a-40a3-b8f3-8613d5750ef4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:50.942293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895389b8-3410-4120-b03c-7e97ad3acd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:50.943024 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895389b8-3410-4120-b03c-7e97ad3acd30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:50.943287 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895389b8-3410-4120-b03c-7e97ad3acd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:50.943775 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895389b8-3410-4120-b03c-7e97ad3acd30 admin admin] PATH_INFO: `/v3/projects/5fcf85e8b32f40f888f4bde52b7e1a8e/users/567966b846814025b553ed2d00f9db6f/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:51.008200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895389b8-3410-4120-b03c-7e97ad3acd30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fcf85e8b32f40f888f4bde52b7e1a8e, user_id=567966b846814025b553ed2d00f9db6f, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:51.018080 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895389b8-3410-4120-b03c-7e97ad3acd30 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:51.061215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1097/2225] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:36:50 2022] PUT /identity/v3/projects/5fcf85e8b32f40f888f4bde52b7e1a8e/users/567966b846814025b553ed2d00f9db6f/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:51.069128 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5b9db-0374-4fd1-a534-c1b0ef6c9365 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:51.092448 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5b9db-0374-4fd1-a534-c1b0ef6c9365 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:51.212691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895389b8-3410-4120-b03c-7e97ad3acd30 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:51.288439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ee1d5-d279-44cd-a69b-2c910a64295b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:51.290496 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ee1d5-d279-44cd-a69b-2c910a64295b admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:51.291191 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ee1d5-d279-44cd-a69b-2c910a64295b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:51.291834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ee1d5-d279-44cd-a69b-2c910a64295b admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:51.295023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ee1d5-d279-44cd-a69b-2c910a64295b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:51.300427 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ee1d5-d279-44cd-a69b-2c910a64295b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:51.330558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1129/2226] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:36:51.334247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ee1d5-d279-44cd-a69b-2c910a64295b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:51.360385 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ee1d5-d279-44cd-a69b-2c910a64295b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:51.369401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84eb88-73d3-420c-ba2c-cb836e59940f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:51.371242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84eb88-73d3-420c-ba2c-cb836e59940f admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:51.371817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84eb88-73d3-420c-ba2c-cb836e59940f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:51.372824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84eb88-73d3-420c-ba2c-cb836e59940f admin admin] PATH_INFO: `/v3/projects/5fcf85e8b32f40f888f4bde52b7e1a8e/users/567966b846814025b553ed2d00f9db6f/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:51.376702 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84eb88-73d3-420c-ba2c-cb836e59940f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fcf85e8b32f40f888f4bde52b7e1a8e, user_id=567966b846814025b553ed2d00f9db6f, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:51.380276 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84eb88-73d3-420c-ba2c-cb836e59940f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:51.401301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1130/2227] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:36:51 2022] PUT /identity/v3/projects/5fcf85e8b32f40f888f4bde52b7e1a8e/users/567966b846814025b553ed2d00f9db6f/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:51.426965 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895389b8-3410-4120-b03c-7e97ad3acd30 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:51.458631 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:51.677977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ac188-888c-45e7-be4b-f8993b666d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:51.679741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ac188-888c-45e7-be4b-f8993b666d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:51.682084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ac188-888c-45e7-be4b-f8993b666d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:51.682784 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ac188-888c-45e7-be4b-f8993b666d3d admin admin] PATH_INFO: `/v3/users/ec70a884d9dd44118a78c90c2b90b421` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:51.699654 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ac188-888c-45e7-be4b-f8993b666d3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec70a884d9dd44118a78c90c2b90b421)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:51.712683 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ac188-888c-45e7-be4b-f8993b666d3d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:51.851994 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e06ac188-888c-45e7-be4b-f8993b666d3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec70a884d9dd44118a78c90c2b90b421'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:51.853045 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e06ac188-888c-45e7-be4b-f8993b666d3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec70a884d9dd44118a78c90c2b90b421'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:51.870314 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e06ac188-888c-45e7-be4b-f8993b666d3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec70a884d9dd44118a78c90c2b90b421'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:51.899055 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e06ac188-888c-45e7-be4b-f8993b666d3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec70a884d9dd44118a78c90c2b90b421'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:51.900313 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e06ac188-888c-45e7-be4b-f8993b666d3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec70a884d9dd44118a78c90c2b90b421'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:51.922708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1098/2228] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:36:51 2022] DELETE /identity/v3/users/ec70a884d9dd44118a78c90c2b90b421 => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:51.925645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e57ee-8ba5-4173-ad04-7bb12329717d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:51.926650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e57ee-8ba5-4173-ad04-7bb12329717d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:51.927179 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e57ee-8ba5-4173-ad04-7bb12329717d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:51.971478 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-b73e57ee-8ba5-4173-ad04-7bb12329717d None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:51.974311 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b73e57ee-8ba5-4173-ad04-7bb12329717d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:52.118126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:52.202471 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1131/2229] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:52.217557 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ac188-888c-45e7-be4b-f8993b666d3d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:52.219375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e06ac188-888c-45e7-be4b-f8993b666d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:52.470286 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ac188-888c-45e7-be4b-f8993b666d3d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:52.507012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3e762-0307-48ae-894d-3c3a58e442d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:52.508359 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3e762-0307-48ae-894d-3c3a58e442d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:52.508678 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3e762-0307-48ae-894d-3c3a58e442d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:52.508958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3e762-0307-48ae-894d-3c3a58e442d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:52.558995 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3e762-0307-48ae-894d-3c3a58e442d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:52.560706 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3e762-0307-48ae-894d-3c3a58e442d5 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:52.598785 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1099/2230] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:36:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:36:53.168707 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87b5a1-94c3-4f46-9657-2ded7dd464f9 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:53.169167 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87b5a1-94c3-4f46-9657-2ded7dd464f9 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:53.169263 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87b5a1-94c3-4f46-9657-2ded7dd464f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:53.238517 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-6d87b5a1-94c3-4f46-9657-2ded7dd464f9 None None] MFA Rules not processed for user `5f0b12ba782244d7b44783c3f3532bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:53.246880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6d87b5a1-94c3-4f46-9657-2ded7dd464f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:53.466568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1132/2231] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:54.467510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3e762-0307-48ae-894d-3c3a58e442d5 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:54.469905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-33b3e762-0307-48ae-894d-3c3a58e442d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:54.500205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3e762-0307-48ae-894d-3c3a58e442d5 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:54.529513 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582986b-d60d-41e9-8a76-4866e8b6a686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:54.531040 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582986b-d60d-41e9-8a76-4866e8b6a686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:54.531333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582986b-d60d-41e9-8a76-4866e8b6a686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:54.531614 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582986b-d60d-41e9-8a76-4866e8b6a686 admin admin] PATH_INFO: `/v3/projects/2d1dff1a858a43fa878dfbfe7a6a8ce1` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:54.537461 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582986b-d60d-41e9-8a76-4866e8b6a686 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d1dff1a858a43fa878dfbfe7a6a8ce1)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:54.541281 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582986b-d60d-41e9-8a76-4866e8b6a686 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:54.665143 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:54.710756 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1582986b-d60d-41e9-8a76-4866e8b6a686 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d1dff1a858a43fa878dfbfe7a6a8ce1', 'request_id': 'req-1582986b-d60d-41e9-8a76-4866e8b6a686'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:54.725241 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1582986b-d60d-41e9-8a76-4866e8b6a686 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d1dff1a858a43fa878dfbfe7a6a8ce1', 'request_id': 'req-1582986b-d60d-41e9-8a76-4866e8b6a686'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:54.743456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1582986b-d60d-41e9-8a76-4866e8b6a686 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d1dff1a858a43fa878dfbfe7a6a8ce1', 'request_id': 'req-1582986b-d60d-41e9-8a76-4866e8b6a686'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:54.773813 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1582986b-d60d-41e9-8a76-4866e8b6a686 admin admin] The token cache is being invalidate because project 2d1dff1a858a43fa878dfbfe7a6a8ce1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:36:54.774729 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1582986b-d60d-41e9-8a76-4866e8b6a686 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:54.776122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1582986b-d60d-41e9-8a76-4866e8b6a686 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:36:54.782851 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1100/2232] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:36:54 2022] DELETE /identity/v3/projects/2d1dff1a858a43fa878dfbfe7a6a8ce1 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:54.788780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89984f-b2ae-41b0-b99e-51a9b7c4dba4 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:54.791643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89984f-b2ae-41b0-b99e-51a9b7c4dba4 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:54.793565 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89984f-b2ae-41b0-b99e-51a9b7c4dba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:54.852286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-6e89984f-b2ae-41b0-b99e-51a9b7c4dba4 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:54.854140 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6e89984f-b2ae-41b0-b99e-51a9b7c4dba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:55.169289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2592de-f0ac-4977-957f-5ec27fa1bf5e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:55.169837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2592de-f0ac-4977-957f-5ec27fa1bf5e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:55.170295 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2592de-f0ac-4977-957f-5ec27fa1bf5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:55.180168 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1133/2233] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:55.231473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6b2592de-f0ac-4977-957f-5ec27fa1bf5e None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:55.236403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6b2592de-f0ac-4977-957f-5ec27fa1bf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:55.266249 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89984f-b2ae-41b0-b99e-51a9b7c4dba4 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:55.278523 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1101/2234] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:55.285477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89984f-b2ae-41b0-b99e-51a9b7c4dba4 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:55.317629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105bf00-f1a2-40c4-b6ff-6ad5c81afb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:55.319676 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105bf00-f1a2-40c4-b6ff-6ad5c81afb70 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:55.325303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105bf00-f1a2-40c4-b6ff-6ad5c81afb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:55.326217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105bf00-f1a2-40c4-b6ff-6ad5c81afb70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:55.327728 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a105bf00-f1a2-40c4-b6ff-6ad5c81afb70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:36:55.328482 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a105bf00-f1a2-40c4-b6ff-6ad5c81afb70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:55.332494 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a105bf00-f1a2-40c4-b6ff-6ad5c81afb70 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:55.363687 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2592de-f0ac-4977-957f-5ec27fa1bf5e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:55.376138 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1134/2235] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:36:55.378413 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2592de-f0ac-4977-957f-5ec27fa1bf5e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:55.388597 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aeda73-4eba-458f-ad78-206771e14c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:55.401390 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeda73-4eba-458f-ad78-206771e14c53 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:55.402760 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeda73-4eba-458f-ad78-206771e14c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:55.406970 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeda73-4eba-458f-ad78-206771e14c53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:55.408565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aeda73-4eba-458f-ad78-206771e14c53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:55.412843 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aeda73-4eba-458f-ad78-206771e14c53 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:55.499279 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105bf00-f1a2-40c4-b6ff-6ad5c81afb70 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:55.508232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1102/2236] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:36:55 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:36:55.518149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105bf00-f1a2-40c4-b6ff-6ad5c81afb70 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:55.612563 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5e2c6-d363-495f-8b03-a50c6a072489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:55.613590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5e2c6-d363-495f-8b03-a50c6a072489 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:55.614091 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5e2c6-d363-495f-8b03-a50c6a072489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:55.614521 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5e2c6-d363-495f-8b03-a50c6a072489 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:55.615416 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5e2c6-d363-495f-8b03-a50c6a072489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:55.617721 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5e2c6-d363-495f-8b03-a50c6a072489 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:55.630693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:55.654678 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1135/2237] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:36:55 2022] POST /identity/v3/users => generated 283 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:36:55.659781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aeda73-4eba-458f-ad78-206771e14c53 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:55.678982 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aeda73-4eba-458f-ad78-206771e14c53 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:55.686496 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1ae11-b966-4a80-90aa-b579bb31e73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:55.687645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1ae11-b966-4a80-90aa-b579bb31e73a admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:55.688127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1ae11-b966-4a80-90aa-b579bb31e73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:55.688842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1ae11-b966-4a80-90aa-b579bb31e73a admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:55.690074 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1ae11-b966-4a80-90aa-b579bb31e73a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:55.692270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1ae11-b966-4a80-90aa-b579bb31e73a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:55.754906 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1103/2238] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:36:55.758815 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5e2c6-d363-495f-8b03-a50c6a072489 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:55.776147 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5e2c6-d363-495f-8b03-a50c6a072489 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:55.790451 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f1b04-2c2e-4a62-a56a-9d546d4d38af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:55.791339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f1b04-2c2e-4a62-a56a-9d546d4d38af admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:55.791950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f1b04-2c2e-4a62-a56a-9d546d4d38af admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:55.796132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f1b04-2c2e-4a62-a56a-9d546d4d38af admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:55.797840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f1b04-2c2e-4a62-a56a-9d546d4d38af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:55.803731 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f1b04-2c2e-4a62-a56a-9d546d4d38af admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:55.845609 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1136/2239] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:36:55.846959 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1ae11-b966-4a80-90aa-b579bb31e73a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:55.868687 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1ae11-b966-4a80-90aa-b579bb31e73a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:55.896678 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98203f0a-df70-47df-94ac-adeec50d537c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:55.897397 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98203f0a-df70-47df-94ac-adeec50d537c admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:55.897826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98203f0a-df70-47df-94ac-adeec50d537c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:55.897970 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98203f0a-df70-47df-94ac-adeec50d537c admin admin] PATH_INFO: `/v3/projects/2312ae1165b440398abe09e22055f14f/users/58b29db6ecea4b86b0abec4083e4ea23/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:55.956254 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98203f0a-df70-47df-94ac-adeec50d537c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2312ae1165b440398abe09e22055f14f, user_id=58b29db6ecea4b86b0abec4083e4ea23, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:55.966689 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98203f0a-df70-47df-94ac-adeec50d537c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:55.997312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1104/2240] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:36:55 2022] PUT /identity/v3/projects/2312ae1165b440398abe09e22055f14f/users/58b29db6ecea4b86b0abec4083e4ea23/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:36:56.013197 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f72669-b048-4403-a2b1-dee14775bcd0 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:56.013640 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f72669-b048-4403-a2b1-dee14775bcd0 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:56.013984 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f72669-b048-4403-a2b1-dee14775bcd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:56.101582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a4f72669-b048-4403-a2b1-dee14775bcd0 None None] MFA Rules not processed for user `58b29db6ecea4b86b0abec4083e4ea23`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:36:56.103144 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a4f72669-b048-4403-a2b1-dee14775bcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:56.249670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1137/2241] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:36:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:36:56.275631 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98203f0a-df70-47df-94ac-adeec50d537c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:56.296554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-98203f0a-df70-47df-94ac-adeec50d537c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:56.461076 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98203f0a-df70-47df-94ac-adeec50d537c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:56.469316 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26bb1f5-32fd-4319-948b-176694fa8c88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:56.470111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bb1f5-32fd-4319-948b-176694fa8c88 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:56.470685 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bb1f5-32fd-4319-948b-176694fa8c88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:56.471237 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bb1f5-32fd-4319-948b-176694fa8c88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:56.488266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26bb1f5-32fd-4319-948b-176694fa8c88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:56.491584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26bb1f5-32fd-4319-948b-176694fa8c88 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:56.520168 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1105/2242] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:36:56 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:37:02.409441 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d478a5-cc30-4f8d-806a-4ee48092988c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:02.410040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d478a5-cc30-4f8d-806a-4ee48092988c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:02.410195 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d478a5-cc30-4f8d-806a-4ee48092988c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:02.517995 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-29d478a5-cc30-4f8d-806a-4ee48092988c None None] MFA Rules not processed for user `567966b846814025b553ed2d00f9db6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:37:02.519585 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-29d478a5-cc30-4f8d-806a-4ee48092988c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:02.727204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1138/2243] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:37:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:37:02.744931 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26bb1f5-32fd-4319-948b-176694fa8c88 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:02.747608 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d26bb1f5-32fd-4319-948b-176694fa8c88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:02.860873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26bb1f5-32fd-4319-948b-176694fa8c88 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:02.865049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe382a9-49d6-49d1-84fb-6140aac45c78 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69d216a568b546589e854c2b4476efe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:02.866223 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe382a9-49d6-49d1-84fb-6140aac45c78 service glance] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:02.866593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe382a9-49d6-49d1-84fb-6140aac45c78 service glance] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:02.867098 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe382a9-49d6-49d1-84fb-6140aac45c78 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:02.869305 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-efe382a9-49d6-49d1-84fb-6140aac45c78 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:02.896289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:37:03.002437 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe382a9-49d6-49d1-84fb-6140aac45c78 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:03.005626 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe382a9-49d6-49d1-84fb-6140aac45c78 service glance] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:03.027903 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1106/2244] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:37:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:37:03.257034 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a6171-2705-4dbb-bde8-28116abd60be None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:03.257307 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a6171-2705-4dbb-bde8-28116abd60be None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:03.257801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a6171-2705-4dbb-bde8-28116abd60be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:03.281615 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-769a6171-2705-4dbb-bde8-28116abd60be None None] MFA Rules not processed for user `f36c6987ef774a1684c55f885e2a4f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:37:03.286385 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-769a6171-2705-4dbb-bde8-28116abd60be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:03.367943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1139/2245] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Aug 1 15:37:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:37:03.407156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe382a9-49d6-49d1-84fb-6140aac45c78 service glance] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:03.411336 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-efe382a9-49d6-49d1-84fb-6140aac45c78 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:03.593844 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe382a9-49d6-49d1-84fb-6140aac45c78 service glance] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:03.602345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a7825-1e6a-488c-be44-827977e4e110 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6873a3e440bd4b63b0082fc85beef49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:03.603743 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a7825-1e6a-488c-be44-827977e4e110 service swift] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:03.604190 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a7825-1e6a-488c-be44-827977e4e110 service swift] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:03.604656 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a7825-1e6a-488c-be44-827977e4e110 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:03.625283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a7825-1e6a-488c-be44-827977e4e110 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:03.627075 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a7825-1e6a-488c-be44-827977e4e110 service swift] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:03.654172 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1107/2246] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Aug 1 15:37:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 15:37:08.515649 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5076981-89b6-445a-85a0-382bb70fcf87 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:08.516299 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5076981-89b6-445a-85a0-382bb70fcf87 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:08.516719 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5076981-89b6-445a-85a0-382bb70fcf87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:08.540324 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:37:08.596495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-b5076981-89b6-445a-85a0-382bb70fcf87 None None] MFA Rules not processed for user `77cb30fae1184064a8c4f822d4d31096`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:37:08.597780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b5076981-89b6-445a-85a0-382bb70fcf87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:08.741998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1140/2247] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:37:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:37:13.638778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=54 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:37:17.724530 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214049c-cf32-4666-8c4b-6ad18ee3529f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:17.725159 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214049c-cf32-4666-8c4b-6ad18ee3529f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:17.725159 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214049c-cf32-4666-8c4b-6ad18ee3529f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:17.744699 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-9214049c-cf32-4666-8c4b-6ad18ee3529f None None] MFA Rules not processed for user `a26637889378427eb884eed624606fd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:37:17.745946 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9214049c-cf32-4666-8c4b-6ad18ee3529f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:17.753048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1108/2248] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:37:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:37:18.725688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=23 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:37:20.592487 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512ae71-e26c-4d91-a1be-01f2806b6677 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:20.593611 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512ae71-e26c-4d91-a1be-01f2806b6677 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:20.593611 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512ae71-e26c-4d91-a1be-01f2806b6677 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:20.595161 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1141/2249] 10.222.0.83 () {54 vars in 921 bytes} [Mon Aug 1 15:37:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:37:20.602560 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297d191-e625-4251-96eb-d3b8126d1386 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:20.603091 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297d191-e625-4251-96eb-d3b8126d1386 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:20.603091 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297d191-e625-4251-96eb-d3b8126d1386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:20.626032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6297d191-e625-4251-96eb-d3b8126d1386 None None] MFA Rules not processed for user `25dc41ffc41c47fdbeec06110d04610c`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:37:20.627270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6297d191-e625-4251-96eb-d3b8126d1386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:20.784228 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1109/2250] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Aug 1 15:37:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:37:20.798562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512ae71-e26c-4d91-a1be-01f2806b6677 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:20.800956 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b512ae71-e26c-4d91-a1be-01f2806b6677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:20.830492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512ae71-e26c-4d91-a1be-01f2806b6677 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:20.836796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16e752-478f-402b-9310-ec205d431e74 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69d216a568b546589e854c2b4476efe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:20.838162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16e752-478f-402b-9310-ec205d431e74 service glance] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:20.839284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16e752-478f-402b-9310-ec205d431e74 service glance] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:20.839726 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16e752-478f-402b-9310-ec205d431e74 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:20.855085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16e752-478f-402b-9310-ec205d431e74 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:20.866173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16e752-478f-402b-9310-ec205d431e74 service glance] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:20.891210 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1142/2251] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:37:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:37:20.975139 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16c4d1-ab8e-41d2-8ccc-5ebd6bab07a8 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:20.975519 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16c4d1-ab8e-41d2-8ccc-5ebd6bab07a8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:20.976032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16c4d1-ab8e-41d2-8ccc-5ebd6bab07a8 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:20.977470 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1110/2252] 10.222.0.83 () {54 vars in 921 bytes} [Mon Aug 1 15:37:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:37:20.985931 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0354ce-2e29-4962-bfe9-12279fa5b9d5 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:20.986710 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0354ce-2e29-4962-bfe9-12279fa5b9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:20.987194 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0354ce-2e29-4962-bfe9-12279fa5b9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:21.011231 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2c0354ce-2e29-4962-bfe9-12279fa5b9d5 None None] MFA Rules not processed for user `76311c7173b442c0af2c745e3e70a07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:37:21.012086 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2c0354ce-2e29-4962-bfe9-12279fa5b9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:21.091993 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1143/2253] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Aug 1 15:37:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:37:21.105431 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16c4d1-ab8e-41d2-8ccc-5ebd6bab07a8 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:21.107471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7e16c4d1-ab8e-41d2-8ccc-5ebd6bab07a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:21.123169 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16c4d1-ab8e-41d2-8ccc-5ebd6bab07a8 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:21.128731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15171aa7-2090-426e-bc50-a68b4f5be93d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:21.129758 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15171aa7-2090-426e-bc50-a68b4f5be93d service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:21.130948 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15171aa7-2090-426e-bc50-a68b4f5be93d service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:21.131446 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15171aa7-2090-426e-bc50-a68b4f5be93d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:21.142443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15171aa7-2090-426e-bc50-a68b4f5be93d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:21.144062 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15171aa7-2090-426e-bc50-a68b4f5be93d service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:21.154757 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1111/2254] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:37:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:37:21.382851 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768a69c-e330-43d5-a6e5-cc8a202df974 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:21.383200 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768a69c-e330-43d5-a6e5-cc8a202df974 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:21.384384 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768a69c-e330-43d5-a6e5-cc8a202df974 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:21.385503 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1144/2255] 10.222.0.83 () {54 vars in 921 bytes} [Mon Aug 1 15:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:37:21.393634 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ad055-a7a1-4e07-9e28-28644ed4eb6c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:21.394049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ad055-a7a1-4e07-9e28-28644ed4eb6c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:21.395206 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ad055-a7a1-4e07-9e28-28644ed4eb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:21.422378 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-b10ad055-a7a1-4e07-9e28-28644ed4eb6c None None] MFA Rules not processed for user `25dc41ffc41c47fdbeec06110d04610c`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:37:21.423390 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b10ad055-a7a1-4e07-9e28-28644ed4eb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:21.435224 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1112/2256] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Aug 1 15:37:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:37:21.458986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768a69c-e330-43d5-a6e5-cc8a202df974 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:21.476309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768a69c-e330-43d5-a6e5-cc8a202df974 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:21.482732 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e17c9f-3702-46ad-8570-0090e35cdbe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:21.483414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e17c9f-3702-46ad-8570-0090e35cdbe2 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:21.483742 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e17c9f-3702-46ad-8570-0090e35cdbe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:21.484025 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e17c9f-3702-46ad-8570-0090e35cdbe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:21.502765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e17c9f-3702-46ad-8570-0090e35cdbe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:21.504259 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e17c9f-3702-46ad-8570-0090e35cdbe2 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:21.522652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1145/2257] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:37:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:37:22.868271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64267b3c-53e5-4bd0-9470-275008a0a623 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:22.869741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64267b3c-53e5-4bd0-9470-275008a0a623 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:22.870499 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64267b3c-53e5-4bd0-9470-275008a0a623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:22.940393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-64267b3c-53e5-4bd0-9470-275008a0a623 None None] MFA Rules not processed for user `f36c6987ef774a1684c55f885e2a4f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:37:22.950993 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-64267b3c-53e5-4bd0-9470-275008a0a623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:22.984121 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1113/2258] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Aug 1 15:37:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:37:23.010125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e17c9f-3702-46ad-8570-0090e35cdbe2 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:23.030428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e17c9f-3702-46ad-8570-0090e35cdbe2 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:23.053326 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752352a3-a732-49ad-8e14-f6ad3582f4d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6873a3e440bd4b63b0082fc85beef49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:23.053326 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752352a3-a732-49ad-8e14-f6ad3582f4d4 service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:23.053628 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752352a3-a732-49ad-8e14-f6ad3582f4d4 service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:23.054212 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752352a3-a732-49ad-8e14-f6ad3582f4d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:23.087161 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752352a3-a732-49ad-8e14-f6ad3582f4d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:23.097942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752352a3-a732-49ad-8e14-f6ad3582f4d4 service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:23.127157 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1146/2259] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Aug 1 15:37:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:37:32.924362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=46 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:37:33.115110 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=41 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:37:43.257094 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e7866-dde4-4cce-9baf-c0b1f7a5e909 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:43.257588 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e7866-dde4-4cce-9baf-c0b1f7a5e909 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:43.257678 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e7866-dde4-4cce-9baf-c0b1f7a5e909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:43.286293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-271e7866-dde4-4cce-9baf-c0b1f7a5e909 None None] MFA Rules not processed for user `f36c6987ef774a1684c55f885e2a4f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:37:43.287646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-271e7866-dde4-4cce-9baf-c0b1f7a5e909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:43.298032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1114/2260] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Aug 1 15:37:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:37:43.311014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752352a3-a732-49ad-8e14-f6ad3582f4d4 service swift] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:43.335362 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752352a3-a732-49ad-8e14-f6ad3582f4d4 service swift] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:43.345825 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408c2ed-c415-4150-b677-165bf377f6a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6873a3e440bd4b63b0082fc85beef49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:43.356538 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408c2ed-c415-4150-b677-165bf377f6a9 service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:43.356912 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408c2ed-c415-4150-b677-165bf377f6a9 service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:43.357268 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408c2ed-c415-4150-b677-165bf377f6a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:43.370517 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b408c2ed-c415-4150-b677-165bf377f6a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:43.372313 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b408c2ed-c415-4150-b677-165bf377f6a9 service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:43.383347 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1147/2261] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Aug 1 15:37:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 15:37:44.544982 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e7866-dde4-4cce-9baf-c0b1f7a5e909 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:44.619121 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e7866-dde4-4cce-9baf-c0b1f7a5e909 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:44.627277 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f4fd1-0796-43cc-b371-08f7865f73d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:44.628227 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f4fd1-0796-43cc-b371-08f7865f73d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:44.628661 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f4fd1-0796-43cc-b371-08f7865f73d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:44.629037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f4fd1-0796-43cc-b371-08f7865f73d8 admin admin] PATH_INFO: `/v3/users/58b29db6ecea4b86b0abec4083e4ea23` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:44.630641 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f4fd1-0796-43cc-b371-08f7865f73d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58b29db6ecea4b86b0abec4083e4ea23)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:44.633515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f4fd1-0796-43cc-b371-08f7865f73d8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:44.702769 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b63f4fd1-0796-43cc-b371-08f7865f73d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58b29db6ecea4b86b0abec4083e4ea23'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:37:44.703454 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b63f4fd1-0796-43cc-b371-08f7865f73d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58b29db6ecea4b86b0abec4083e4ea23'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:37:44.711756 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b63f4fd1-0796-43cc-b371-08f7865f73d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58b29db6ecea4b86b0abec4083e4ea23'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:37:44.725472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b63f4fd1-0796-43cc-b371-08f7865f73d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58b29db6ecea4b86b0abec4083e4ea23'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:37:44.726163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b63f4fd1-0796-43cc-b371-08f7865f73d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58b29db6ecea4b86b0abec4083e4ea23'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:37:44.742093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1115/2262] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:37:44 2022] DELETE /identity/v3/users/58b29db6ecea4b86b0abec4083e4ea23 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:37:44.745197 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff67d3-4348-4311-a176-97239e04e25a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:44.745762 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff67d3-4348-4311-a176-97239e04e25a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:44.746047 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff67d3-4348-4311-a176-97239e04e25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:44.774292 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-61ff67d3-4348-4311-a176-97239e04e25a None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:37:44.775676 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-61ff67d3-4348-4311-a176-97239e04e25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:44.842962 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1148/2263] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:37:44.852718 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f4fd1-0796-43cc-b371-08f7865f73d8 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:44.854211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b63f4fd1-0796-43cc-b371-08f7865f73d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:44.953037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f4fd1-0796-43cc-b371-08f7865f73d8 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:44.956915 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f187bd-44f9-4b57-afc8-1015cdf25d1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:44.957612 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f187bd-44f9-4b57-afc8-1015cdf25d1f service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:44.958003 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f187bd-44f9-4b57-afc8-1015cdf25d1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:44.958239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f187bd-44f9-4b57-afc8-1015cdf25d1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:44.969103 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f187bd-44f9-4b57-afc8-1015cdf25d1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:44.970593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f187bd-44f9-4b57-afc8-1015cdf25d1f service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:44.980535 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1116/2264] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:37:45.329865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff67d3-4348-4311-a176-97239e04e25a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:45.332301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-61ff67d3-4348-4311-a176-97239e04e25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:45.353144 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff67d3-4348-4311-a176-97239e04e25a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:45.358793 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1ed60-d43f-41f2-a5db-fa99bad5a185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:45.359604 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1ed60-d43f-41f2-a5db-fa99bad5a185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:45.359995 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1ed60-d43f-41f2-a5db-fa99bad5a185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:45.360330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1ed60-d43f-41f2-a5db-fa99bad5a185 admin admin] PATH_INFO: `/v3/projects/2312ae1165b440398abe09e22055f14f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:45.361625 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1ed60-d43f-41f2-a5db-fa99bad5a185 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2312ae1165b440398abe09e22055f14f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:45.363741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1ed60-d43f-41f2-a5db-fa99bad5a185 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:45.445776 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-adf1ed60-d43f-41f2-a5db-fa99bad5a185 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2312ae1165b440398abe09e22055f14f', 'request_id': 'req-adf1ed60-d43f-41f2-a5db-fa99bad5a185'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:37:45.465585 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-adf1ed60-d43f-41f2-a5db-fa99bad5a185 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2312ae1165b440398abe09e22055f14f', 'request_id': 'req-adf1ed60-d43f-41f2-a5db-fa99bad5a185'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:37:45.465585 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-adf1ed60-d43f-41f2-a5db-fa99bad5a185 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2312ae1165b440398abe09e22055f14f', 'request_id': 'req-adf1ed60-d43f-41f2-a5db-fa99bad5a185'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:37:45.477931 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-adf1ed60-d43f-41f2-a5db-fa99bad5a185 admin admin] The token cache is being invalidate because project 2312ae1165b440398abe09e22055f14f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:37:45.478355 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-adf1ed60-d43f-41f2-a5db-fa99bad5a185 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:37:45.479161 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-adf1ed60-d43f-41f2-a5db-fa99bad5a185 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:37:45.484461 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1149/2265] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:37:45 2022] DELETE /identity/v3/projects/2312ae1165b440398abe09e22055f14f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:37:45.486567 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb584601-7f77-4f37-89ec-a50e2d3f2b0f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:45.487000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb584601-7f77-4f37-89ec-a50e2d3f2b0f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:45.487358 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb584601-7f77-4f37-89ec-a50e2d3f2b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:45.514163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-bb584601-7f77-4f37-89ec-a50e2d3f2b0f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:37:45.515362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bb584601-7f77-4f37-89ec-a50e2d3f2b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:45.524902 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:37:45.580732 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1117/2266] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:37:45.590334 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01405fc9-dcca-4d4e-a4b2-6e86aec62fd4 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:45.590562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01405fc9-dcca-4d4e-a4b2-6e86aec62fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:45.590820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01405fc9-dcca-4d4e-a4b2-6e86aec62fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:45.613133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-01405fc9-dcca-4d4e-a4b2-6e86aec62fd4 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:37:45.614689 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-01405fc9-dcca-4d4e-a4b2-6e86aec62fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:45.627562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1150/2267] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:37:45.628533 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb584601-7f77-4f37-89ec-a50e2d3f2b0f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:45.638609 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb584601-7f77-4f37-89ec-a50e2d3f2b0f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:45.643564 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd9b68-db80-40c7-b5d8-d0d027d99d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:45.644189 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd9b68-db80-40c7-b5d8-d0d027d99d9d admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:45.644469 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd9b68-db80-40c7-b5d8-d0d027d99d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:45.644800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd9b68-db80-40c7-b5d8-d0d027d99d9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:45.645543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd9b68-db80-40c7-b5d8-d0d027d99d9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:37:45.645850 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd9b68-db80-40c7-b5d8-d0d027d99d9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:45.647471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd9b68-db80-40c7-b5d8-d0d027d99d9d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:45.659416 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1118/2268] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:37:45.663717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01405fc9-dcca-4d4e-a4b2-6e86aec62fd4 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:45.675058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01405fc9-dcca-4d4e-a4b2-6e86aec62fd4 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:45.679682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f26e7b-ad47-4658-a808-2eea5368976c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:45.680319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f26e7b-ad47-4658-a808-2eea5368976c admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:45.680585 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f26e7b-ad47-4658-a808-2eea5368976c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:45.680850 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f26e7b-ad47-4658-a808-2eea5368976c admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:45.681613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f26e7b-ad47-4658-a808-2eea5368976c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:45.683611 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f26e7b-ad47-4658-a808-2eea5368976c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:45.749096 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1151/2269] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:37:45 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:37:45.749790 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd9b68-db80-40c7-b5d8-d0d027d99d9d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:45.763750 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd9b68-db80-40c7-b5d8-d0d027d99d9d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:45.841537 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4d773-910b-4106-be24-47cc67f64d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:45.842132 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4d773-910b-4106-be24-47cc67f64d52 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:45.842416 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4d773-910b-4106-be24-47cc67f64d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:45.842696 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4d773-910b-4106-be24-47cc67f64d52 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:45.843385 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f4d773-910b-4106-be24-47cc67f64d52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:45.859836 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f4d773-910b-4106-be24-47cc67f64d52 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:45.912398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1119/2270] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:37:45 2022] POST /identity/v3/users => generated 287 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:37:45.916731 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f26e7b-ad47-4658-a808-2eea5368976c admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:45.928879 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f26e7b-ad47-4658-a808-2eea5368976c admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:45.935176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b3000-bf3b-4b88-9620-d46f3c20f1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:45.935983 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b3000-bf3b-4b88-9620-d46f3c20f1a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:45.936119 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b3000-bf3b-4b88-9620-d46f3c20f1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:45.936393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b3000-bf3b-4b88-9620-d46f3c20f1a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:45.937103 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b3000-bf3b-4b88-9620-d46f3c20f1a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:45.938736 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b3000-bf3b-4b88-9620-d46f3c20f1a6 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:45.976973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1152/2271] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:37:45.978200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4d773-910b-4106-be24-47cc67f64d52 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:46.011062 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4d773-910b-4106-be24-47cc67f64d52 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:46.021860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23121e4-5b3f-4b80-8827-ae1b09ea5b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:46.022536 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23121e4-5b3f-4b80-8827-ae1b09ea5b96 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:46.022814 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23121e4-5b3f-4b80-8827-ae1b09ea5b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:46.023090 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23121e4-5b3f-4b80-8827-ae1b09ea5b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:46.023804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23121e4-5b3f-4b80-8827-ae1b09ea5b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:46.025485 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23121e4-5b3f-4b80-8827-ae1b09ea5b96 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:46.044169 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1120/2272] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:37:46.045439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b3000-bf3b-4b88-9620-d46f3c20f1a6 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:46.055839 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b3000-bf3b-4b88-9620-d46f3c20f1a6 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:46.060484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8862ba-0d19-4cd4-9c81-751b18666717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:46.061151 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8862ba-0d19-4cd4-9c81-751b18666717 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:46.061428 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8862ba-0d19-4cd4-9c81-751b18666717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:46.061697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8862ba-0d19-4cd4-9c81-751b18666717 admin admin] PATH_INFO: `/v3/projects/f1c097fae5f24937813a4d71a228025a/users/ffdcf6d5058b45b6bc498ca94ceab696/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:46.076844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8862ba-0d19-4cd4-9c81-751b18666717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1c097fae5f24937813a4d71a228025a, user_id=ffdcf6d5058b45b6bc498ca94ceab696, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:46.079518 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8862ba-0d19-4cd4-9c81-751b18666717 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:46.101949 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1153/2273] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:37:46 2022] PUT /identity/v3/projects/f1c097fae5f24937813a4d71a228025a/users/ffdcf6d5058b45b6bc498ca94ceab696/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:37:46.178417 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b01694-6810-4bcf-b30c-f8ec56de1186 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:46.178847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b01694-6810-4bcf-b30c-f8ec56de1186 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:46.179197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b01694-6810-4bcf-b30c-f8ec56de1186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:46.209245 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-35b01694-6810-4bcf-b30c-f8ec56de1186 None None] MFA Rules not processed for user `ffdcf6d5058b45b6bc498ca94ceab696`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:37:46.210532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-35b01694-6810-4bcf-b30c-f8ec56de1186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:46.288735 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1121/2274] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:37:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:37:46.300819 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8862ba-0d19-4cd4-9c81-751b18666717 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:46.302646 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3b8862ba-0d19-4cd4-9c81-751b18666717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:46.313848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:37:46.380703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8862ba-0d19-4cd4-9c81-751b18666717 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:46.384733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af1143-36c4-417a-aec4-7e9696495c42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:46.385876 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af1143-36c4-417a-aec4-7e9696495c42 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:46.387057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af1143-36c4-417a-aec4-7e9696495c42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:46.387439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af1143-36c4-417a-aec4-7e9696495c42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:46.397967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af1143-36c4-417a-aec4-7e9696495c42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:46.399579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af1143-36c4-417a-aec4-7e9696495c42 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:46.410438 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1154/2275] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:37:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:37:46.435288 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b01694-6810-4bcf-b30c-f8ec56de1186 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:46.448509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:37:46.534022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b01694-6810-4bcf-b30c-f8ec56de1186 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:46.544928 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ee4d5-787e-4fb3-9b39-0f0a42c4a4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:46.545870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ee4d5-787e-4fb3-9b39-0f0a42c4a4da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:46.546183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ee4d5-787e-4fb3-9b39-0f0a42c4a4da admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:46.546464 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ee4d5-787e-4fb3-9b39-0f0a42c4a4da admin admin] PATH_INFO: `/v3/users/ffdcf6d5058b45b6bc498ca94ceab696` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:46.547682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ee4d5-787e-4fb3-9b39-0f0a42c4a4da admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffdcf6d5058b45b6bc498ca94ceab696)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:46.549709 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ee4d5-787e-4fb3-9b39-0f0a42c4a4da admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:46.611164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8a4ee4d5-787e-4fb3-9b39-0f0a42c4a4da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffdcf6d5058b45b6bc498ca94ceab696'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:37:46.611875 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8a4ee4d5-787e-4fb3-9b39-0f0a42c4a4da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffdcf6d5058b45b6bc498ca94ceab696'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:37:46.621666 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8a4ee4d5-787e-4fb3-9b39-0f0a42c4a4da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffdcf6d5058b45b6bc498ca94ceab696'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:37:46.635630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8a4ee4d5-787e-4fb3-9b39-0f0a42c4a4da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffdcf6d5058b45b6bc498ca94ceab696'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:37:46.636889 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8a4ee4d5-787e-4fb3-9b39-0f0a42c4a4da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffdcf6d5058b45b6bc498ca94ceab696'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:37:46.649710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1122/2276] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:37:46 2022] DELETE /identity/v3/users/ffdcf6d5058b45b6bc498ca94ceab696 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:37:46.655360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6e845-902c-4de9-a7de-c0d6625f5c17 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:46.661740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6e845-902c-4de9-a7de-c0d6625f5c17 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:46.661740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6e845-902c-4de9-a7de-c0d6625f5c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:46.681959 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-73b6e845-902c-4de9-a7de-c0d6625f5c17 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:37:46.684006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-73b6e845-902c-4de9-a7de-c0d6625f5c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:46.763305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1155/2277] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:37:46.773370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ee4d5-787e-4fb3-9b39-0f0a42c4a4da admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:46.774915 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8a4ee4d5-787e-4fb3-9b39-0f0a42c4a4da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:46.838068 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ee4d5-787e-4fb3-9b39-0f0a42c4a4da admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:46.842080 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096885f8-2cc8-4552-abb2-7fbc8eb357f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:46.842731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096885f8-2cc8-4552-abb2-7fbc8eb357f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:46.842985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096885f8-2cc8-4552-abb2-7fbc8eb357f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:46.843256 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096885f8-2cc8-4552-abb2-7fbc8eb357f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:46.853060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096885f8-2cc8-4552-abb2-7fbc8eb357f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:46.854580 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096885f8-2cc8-4552-abb2-7fbc8eb357f2 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:46.865677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1123/2278] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:37:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:37:47.219296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6e845-902c-4de9-a7de-c0d6625f5c17 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:47.220877 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-73b6e845-902c-4de9-a7de-c0d6625f5c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:47.237259 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6e845-902c-4de9-a7de-c0d6625f5c17 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:47.257265 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078d989-6d73-4156-8a09-6e1b38a14578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:47.258064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078d989-6d73-4156-8a09-6e1b38a14578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:47.258392 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078d989-6d73-4156-8a09-6e1b38a14578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:47.258751 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078d989-6d73-4156-8a09-6e1b38a14578 admin admin] PATH_INFO: `/v3/projects/f1c097fae5f24937813a4d71a228025a` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:47.260065 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078d989-6d73-4156-8a09-6e1b38a14578 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1c097fae5f24937813a4d71a228025a)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:47.262512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078d989-6d73-4156-8a09-6e1b38a14578 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:47.357226 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a078d989-6d73-4156-8a09-6e1b38a14578 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1c097fae5f24937813a4d71a228025a', 'request_id': 'req-a078d989-6d73-4156-8a09-6e1b38a14578'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:37:47.363833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a078d989-6d73-4156-8a09-6e1b38a14578 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1c097fae5f24937813a4d71a228025a', 'request_id': 'req-a078d989-6d73-4156-8a09-6e1b38a14578'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:37:47.370696 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a078d989-6d73-4156-8a09-6e1b38a14578 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1c097fae5f24937813a4d71a228025a', 'request_id': 'req-a078d989-6d73-4156-8a09-6e1b38a14578'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:37:47.385009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a078d989-6d73-4156-8a09-6e1b38a14578 admin admin] The token cache is being invalidate because project f1c097fae5f24937813a4d71a228025a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:37:47.385320 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a078d989-6d73-4156-8a09-6e1b38a14578 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:37:47.385989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a078d989-6d73-4156-8a09-6e1b38a14578 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:37:47.387607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1156/2279] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:37:47 2022] DELETE /identity/v3/projects/f1c097fae5f24937813a4d71a228025a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:37:47.392652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f3137-9093-49e4-8741-71c0bbb77caf None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:47.392939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f3137-9093-49e4-8741-71c0bbb77caf None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:47.393218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f3137-9093-49e4-8741-71c0bbb77caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:47.413299 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-673f3137-9093-49e4-8741-71c0bbb77caf None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:37:47.414507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-673f3137-9093-49e4-8741-71c0bbb77caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:47.458194 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:37:47.520433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1124/2280] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:37:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:37:47.535391 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cd900-17b7-433a-a531-1997b3bedf66 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:47.536140 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cd900-17b7-433a-a531-1997b3bedf66 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:47.536533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cd900-17b7-433a-a531-1997b3bedf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:47.579407 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-b85cd900-17b7-433a-a531-1997b3bedf66 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:37:47.580735 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b85cd900-17b7-433a-a531-1997b3bedf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:47.590802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1157/2281] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:37:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:37:47.596356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f3137-9093-49e4-8741-71c0bbb77caf None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:47.611407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f3137-9093-49e4-8741-71c0bbb77caf None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:47.620743 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7d51b-6488-4c23-a44d-3c392cfa76d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:47.621489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7d51b-6488-4c23-a44d-3c392cfa76d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:47.621824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7d51b-6488-4c23-a44d-3c392cfa76d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:47.622338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7d51b-6488-4c23-a44d-3c392cfa76d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:47.623163 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7d51b-6488-4c23-a44d-3c392cfa76d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:37:47.623479 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7d51b-6488-4c23-a44d-3c392cfa76d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:47.625471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7d51b-6488-4c23-a44d-3c392cfa76d2 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:47.642344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1125/2282] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 15:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:37:47.644807 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cd900-17b7-433a-a531-1997b3bedf66 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:47.654812 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cd900-17b7-433a-a531-1997b3bedf66 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:47.659826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba01dbc-0d85-4ba4-8190-82148b9fd448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:47.660734 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba01dbc-0d85-4ba4-8190-82148b9fd448 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:47.661085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba01dbc-0d85-4ba4-8190-82148b9fd448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:47.661424 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba01dbc-0d85-4ba4-8190-82148b9fd448 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:47.662154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba01dbc-0d85-4ba4-8190-82148b9fd448 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:47.664488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba01dbc-0d85-4ba4-8190-82148b9fd448 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:47.706319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1158/2283] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 15:37:47 2022] POST /identity/v3/projects => generated 389 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:37:47.707621 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7d51b-6488-4c23-a44d-3c392cfa76d2 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:47.725245 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7d51b-6488-4c23-a44d-3c392cfa76d2 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:47.818408 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5ed5c-d04c-41fa-a332-a54c3cc472fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:47.819000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5ed5c-d04c-41fa-a332-a54c3cc472fd admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:47.819283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5ed5c-d04c-41fa-a332-a54c3cc472fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:47.819566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5ed5c-d04c-41fa-a332-a54c3cc472fd admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:47.820259 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5ed5c-d04c-41fa-a332-a54c3cc472fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:47.822369 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5ed5c-d04c-41fa-a332-a54c3cc472fd admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:47.866094 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1126/2284] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 15:37:47 2022] POST /identity/v3/users => generated 295 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:37:47.866701 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba01dbc-0d85-4ba4-8190-82148b9fd448 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:47.873634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:37:47.879055 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba01dbc-0d85-4ba4-8190-82148b9fd448 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:47.887264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706276a4-36a3-4b87-aefb-fe011e641518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:47.888336 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706276a4-36a3-4b87-aefb-fe011e641518 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:47.888871 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706276a4-36a3-4b87-aefb-fe011e641518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:47.888990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706276a4-36a3-4b87-aefb-fe011e641518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:47.889860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706276a4-36a3-4b87-aefb-fe011e641518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:47.892349 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706276a4-36a3-4b87-aefb-fe011e641518 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:47.906663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1159/2285] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:37:47.910677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5ed5c-d04c-41fa-a332-a54c3cc472fd admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:47.921203 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5ed5c-d04c-41fa-a332-a54c3cc472fd admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:47.925881 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a066f-1a43-4c27-9ddf-f5013e705987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:47.926531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a066f-1a43-4c27-9ddf-f5013e705987 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:47.926816 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a066f-1a43-4c27-9ddf-f5013e705987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:47.927092 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a066f-1a43-4c27-9ddf-f5013e705987 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:47.927789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a066f-1a43-4c27-9ddf-f5013e705987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:47.929399 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a066f-1a43-4c27-9ddf-f5013e705987 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:47.942029 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1127/2286] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 15:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:37:47.946327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706276a4-36a3-4b87-aefb-fe011e641518 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:47.957536 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706276a4-36a3-4b87-aefb-fe011e641518 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:47.962688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17cb1b-7e9d-4c66-a6eb-c45a3567800d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:47.963652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17cb1b-7e9d-4c66-a6eb-c45a3567800d admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:47.963967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17cb1b-7e9d-4c66-a6eb-c45a3567800d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:47.964216 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17cb1b-7e9d-4c66-a6eb-c45a3567800d admin admin] PATH_INFO: `/v3/projects/c1c78d23cae448c7ad7771f924f858e4/users/3f56b135708f44218c36ecc39cdda65a/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:47.981539 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17cb1b-7e9d-4c66-a6eb-c45a3567800d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1c78d23cae448c7ad7771f924f858e4, user_id=3f56b135708f44218c36ecc39cdda65a, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:47.984382 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17cb1b-7e9d-4c66-a6eb-c45a3567800d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:48.002973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1160/2287] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 15:37:47 2022] PUT /identity/v3/projects/c1c78d23cae448c7ad7771f924f858e4/users/3f56b135708f44218c36ecc39cdda65a/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:37:48.014259 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c3682-1109-43ea-9cfe-c59555b9d5ac None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:48.014456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c3682-1109-43ea-9cfe-c59555b9d5ac None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:48.014733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c3682-1109-43ea-9cfe-c59555b9d5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:48.042553 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-2e0c3682-1109-43ea-9cfe-c59555b9d5ac None None] MFA Rules not processed for user `3f56b135708f44218c36ecc39cdda65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:37:48.044426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2e0c3682-1109-43ea-9cfe-c59555b9d5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:48.114186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1128/2288] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:37:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:37:48.126070 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17cb1b-7e9d-4c66-a6eb-c45a3567800d admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:37:48.128107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2c17cb1b-7e9d-4c66-a6eb-c45a3567800d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:37:48.189231 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17cb1b-7e9d-4c66-a6eb-c45a3567800d admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:37:48.193600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50b400-5edf-4966-bd17-4f433003631a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:37:48.194289 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50b400-5edf-4966-bd17-4f433003631a service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:37:48.194600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50b400-5edf-4966-bd17-4f433003631a service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:37:48.194912 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50b400-5edf-4966-bd17-4f433003631a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:37:48.204819 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50b400-5edf-4966-bd17-4f433003631a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:37:48.206458 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50b400-5edf-4966-bd17-4f433003631a service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:37:48.228335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1161/2289] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:37:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:37:58.109916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:37:58.213758 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:38:46.388857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c3682-1109-43ea-9cfe-c59555b9d5ac None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:38:46.390432 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2e0c3682-1109-43ea-9cfe-c59555b9d5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:38:46.448177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c3682-1109-43ea-9cfe-c59555b9d5ac None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:38:46.451958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d2429-54b5-44c6-bdf1-47bb11790d76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:38:46.452587 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d2429-54b5-44c6-bdf1-47bb11790d76 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:38:46.452873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d2429-54b5-44c6-bdf1-47bb11790d76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:38:46.453154 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d2429-54b5-44c6-bdf1-47bb11790d76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:38:46.455001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-384d2429-54b5-44c6-bdf1-47bb11790d76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:38:46.528714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d2429-54b5-44c6-bdf1-47bb11790d76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:38:46.530555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d2429-54b5-44c6-bdf1-47bb11790d76 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:38:46.551604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1129/2290] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:38:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:38:55.094971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50b400-5edf-4966-bd17-4f433003631a service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:38:55.104410 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50b400-5edf-4966-bd17-4f433003631a service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:38:55.108300 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc66ab5-0816-43da-8bf0-7705d4f0849b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:38:55.109117 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc66ab5-0816-43da-8bf0-7705d4f0849b service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:38:55.109395 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc66ab5-0816-43da-8bf0-7705d4f0849b service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:38:55.109759 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc66ab5-0816-43da-8bf0-7705d4f0849b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:38:55.111587 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-bfc66ab5-0816-43da-8bf0-7705d4f0849b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:38:55.127625 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc66ab5-0816-43da-8bf0-7705d4f0849b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:38:55.129077 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc66ab5-0816-43da-8bf0-7705d4f0849b service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:38:55.142046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1162/2291] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:38:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:38:56.537633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=54 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:38:58.312494 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d2429-54b5-44c6-bdf1-47bb11790d76 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:38:58.313875 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-384d2429-54b5-44c6-bdf1-47bb11790d76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:38:58.328459 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d2429-54b5-44c6-bdf1-47bb11790d76 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:38:58.332719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685de4f-9ab1-4a13-af34-e1042881a888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:38:58.333574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685de4f-9ab1-4a13-af34-e1042881a888 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:38:58.334624 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685de4f-9ab1-4a13-af34-e1042881a888 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:38:58.335102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685de4f-9ab1-4a13-af34-e1042881a888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:38:58.337243 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a685de4f-9ab1-4a13-af34-e1042881a888 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:38:58.419388 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685de4f-9ab1-4a13-af34-e1042881a888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:38:58.421280 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685de4f-9ab1-4a13-af34-e1042881a888 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:38:58.451836 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1130/2292] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:38:58 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:38:59.315107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc66ab5-0816-43da-8bf0-7705d4f0849b service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:38:59.325230 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc66ab5-0816-43da-8bf0-7705d4f0849b service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:38:59.329604 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2678a9d-7835-463d-8fe6-c5ebce4f1047 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:38:59.330570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2678a9d-7835-463d-8fe6-c5ebce4f1047 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:38:59.331032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2678a9d-7835-463d-8fe6-c5ebce4f1047 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:38:59.331378 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2678a9d-7835-463d-8fe6-c5ebce4f1047 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:38:59.333530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f2678a9d-7835-463d-8fe6-c5ebce4f1047 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:38:59.404732 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2678a9d-7835-463d-8fe6-c5ebce4f1047 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:38:59.406667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2678a9d-7835-463d-8fe6-c5ebce4f1047 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:38:59.419991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1163/2293] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:38:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:39:01.384715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d46935-53e5-4111-bb7d-5b0010be7d5d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:39:01.385454 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d46935-53e5-4111-bb7d-5b0010be7d5d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:39:01.385454 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d46935-53e5-4111-bb7d-5b0010be7d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:39:01.409650 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-79d46935-53e5-4111-bb7d-5b0010be7d5d None None] MFA Rules not processed for user `f36c6987ef774a1684c55f885e2a4f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:39:01.411396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-79d46935-53e5-4111-bb7d-5b0010be7d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:39:01.495893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1131/2294] 10.222.0.83 () {58 vars in 967 bytes} [Mon Aug 1 15:39:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:39:01.546788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2678a9d-7835-463d-8fe6-c5ebce4f1047 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:39:01.548243 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f2678a9d-7835-463d-8fe6-c5ebce4f1047 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:39:01.639404 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2678a9d-7835-463d-8fe6-c5ebce4f1047 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:39:01.645477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56a27f-dd70-437b-964c-9c800f49214d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6873a3e440bd4b63b0082fc85beef49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:39:01.646295 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56a27f-dd70-437b-964c-9c800f49214d service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:39:01.646697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56a27f-dd70-437b-964c-9c800f49214d service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:39:01.647155 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56a27f-dd70-437b-964c-9c800f49214d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:39:01.658144 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56a27f-dd70-437b-964c-9c800f49214d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:39:01.660293 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56a27f-dd70-437b-964c-9c800f49214d service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:39:01.671109 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1164/2295] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Aug 1 15:39:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:39:03.355437 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d46935-53e5-4111-bb7d-5b0010be7d5d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:39:03.356845 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-79d46935-53e5-4111-bb7d-5b0010be7d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:39:03.372782 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d46935-53e5-4111-bb7d-5b0010be7d5d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:39:03.378332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac380932-2f9a-4a61-81d4-ceb62d55010c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:39:03.378949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac380932-2f9a-4a61-81d4-ceb62d55010c service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:39:03.379233 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac380932-2f9a-4a61-81d4-ceb62d55010c service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:39:03.379509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac380932-2f9a-4a61-81d4-ceb62d55010c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:39:03.381194 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ac380932-2f9a-4a61-81d4-ceb62d55010c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:39:03.439084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac380932-2f9a-4a61-81d4-ceb62d55010c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:39:03.440686 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac380932-2f9a-4a61-81d4-ceb62d55010c service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:39:03.450487 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1132/2296] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:39:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:39:11.397831 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56a27f-dd70-437b-964c-9c800f49214d service swift] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:39:11.410135 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56a27f-dd70-437b-964c-9c800f49214d service swift] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:39:11.415522 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81ed96-ff07-42d3-95b1-666993811d57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:39:11.416124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81ed96-ff07-42d3-95b1-666993811d57 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:39:11.416401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81ed96-ff07-42d3-95b1-666993811d57 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:39:11.416669 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81ed96-ff07-42d3-95b1-666993811d57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:39:11.419060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fa81ed96-ff07-42d3-95b1-666993811d57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:39:11.477824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa81ed96-ff07-42d3-95b1-666993811d57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:39:11.479500 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa81ed96-ff07-42d3-95b1-666993811d57 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:39:11.491335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1165/2297] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:39:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:39:13.445477 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=83 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:39:16.580857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac380932-2f9a-4a61-81d4-ceb62d55010c service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:39:16.591910 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac380932-2f9a-4a61-81d4-ceb62d55010c service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:39:16.596243 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf38f3-1f93-427b-95ad-6c307c8d8325 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:39:16.597186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf38f3-1f93-427b-95ad-6c307c8d8325 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:39:16.597556 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf38f3-1f93-427b-95ad-6c307c8d8325 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:39:16.597830 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf38f3-1f93-427b-95ad-6c307c8d8325 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:39:16.599736 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6bbf38f3-1f93-427b-95ad-6c307c8d8325 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:39:16.659818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf38f3-1f93-427b-95ad-6c307c8d8325 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:39:16.661183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf38f3-1f93-427b-95ad-6c307c8d8325 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:39:16.671101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1133/2298] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:39:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:39:21.486607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=94 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:39:26.666035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=28 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:39:58.793507 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81ed96-ff07-42d3-95b1-666993811d57 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:39:58.803411 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81ed96-ff07-42d3-95b1-666993811d57 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:39:58.807107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e094aed-e741-47a2-b711-a0ead1aac9aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:39:58.807715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e094aed-e741-47a2-b711-a0ead1aac9aa service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:39:58.807996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e094aed-e741-47a2-b711-a0ead1aac9aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:39:58.808262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e094aed-e741-47a2-b711-a0ead1aac9aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:39:58.810025 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7e094aed-e741-47a2-b711-a0ead1aac9aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:39:58.819953 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e094aed-e741-47a2-b711-a0ead1aac9aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:39:58.821380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e094aed-e741-47a2-b711-a0ead1aac9aa service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:39:58.829841 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1166/2299] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:39:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:40:08.825847 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=6 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:40:21.288521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf38f3-1f93-427b-95ad-6c307c8d8325 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:40:21.299684 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf38f3-1f93-427b-95ad-6c307c8d8325 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:40:21.303682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e7178-358a-4365-9419-91c6fb5177d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:40:21.304314 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e7178-358a-4365-9419-91c6fb5177d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:40:21.304596 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e7178-358a-4365-9419-91c6fb5177d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:40:21.304869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e7178-358a-4365-9419-91c6fb5177d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:40:21.306678 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-cc4e7178-358a-4365-9419-91c6fb5177d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:40:21.366955 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e7178-358a-4365-9419-91c6fb5177d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:40:21.368389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e7178-358a-4365-9419-91c6fb5177d9 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:40:21.377744 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1134/2300] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:40:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:40:31.373290 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=28 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:41:21.662125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e094aed-e741-47a2-b711-a0ead1aac9aa service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:41:21.675705 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e094aed-e741-47a2-b711-a0ead1aac9aa service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:41:21.681929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c4103-4950-473e-8e40-9b357ac35204 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:41:21.682540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c4103-4950-473e-8e40-9b357ac35204 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:41:21.682826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c4103-4950-473e-8e40-9b357ac35204 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:41:21.683102 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c4103-4950-473e-8e40-9b357ac35204 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:41:21.685226 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b25c4103-4950-473e-8e40-9b357ac35204 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:41:21.749170 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c4103-4950-473e-8e40-9b357ac35204 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:41:21.750823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c4103-4950-473e-8e40-9b357ac35204 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:41:21.760379 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1167/2301] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:41:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:41:23.006837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e7178-358a-4365-9419-91c6fb5177d9 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:41:23.023466 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e7178-358a-4365-9419-91c6fb5177d9 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:41:23.029493 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737f3af-8f25-4acb-b132-9018ddd22a1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:41:23.030353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737f3af-8f25-4acb-b132-9018ddd22a1b service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:41:23.030710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737f3af-8f25-4acb-b132-9018ddd22a1b service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:41:23.031061 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737f3af-8f25-4acb-b132-9018ddd22a1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:41:23.033413 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8737f3af-8f25-4acb-b132-9018ddd22a1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:41:23.046757 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737f3af-8f25-4acb-b132-9018ddd22a1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:41:23.048078 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737f3af-8f25-4acb-b132-9018ddd22a1b service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:41:23.061981 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1135/2302] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:41:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:41:31.756238 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=28 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:41:33.055343 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=6 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:41:49.736925 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c4103-4950-473e-8e40-9b357ac35204 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:41:49.738488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b25c4103-4950-473e-8e40-9b357ac35204 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:41:49.875694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c4103-4950-473e-8e40-9b357ac35204 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:41:49.881880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb223d27-5381-4a27-b7d3-eaeff34dc683 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:41:49.882644 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb223d27-5381-4a27-b7d3-eaeff34dc683 service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:41:49.883054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb223d27-5381-4a27-b7d3-eaeff34dc683 service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:41:49.883392 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb223d27-5381-4a27-b7d3-eaeff34dc683 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:41:49.885472 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fb223d27-5381-4a27-b7d3-eaeff34dc683 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:41:49.898009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb223d27-5381-4a27-b7d3-eaeff34dc683 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:41:49.899663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb223d27-5381-4a27-b7d3-eaeff34dc683 service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:41:49.910566 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1168/2303] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:41:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:41:59.905755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=63 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:42:57.717742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737f3af-8f25-4acb-b132-9018ddd22a1b service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:42:57.755403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737f3af-8f25-4acb-b132-9018ddd22a1b service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:42:57.759219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c0f11-6c33-4edb-9da7-5decf06e6a86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:42:57.759846 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c0f11-6c33-4edb-9da7-5decf06e6a86 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:42:57.760131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c0f11-6c33-4edb-9da7-5decf06e6a86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:42:57.760408 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c0f11-6c33-4edb-9da7-5decf06e6a86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:42:57.762609 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-177c0f11-6c33-4edb-9da7-5decf06e6a86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:42:57.826952 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c0f11-6c33-4edb-9da7-5decf06e6a86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:42:57.828484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c0f11-6c33-4edb-9da7-5decf06e6a86 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:42:57.842704 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1136/2304] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:42:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:42:57.891220 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb223d27-5381-4a27-b7d3-eaeff34dc683 service placement] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:42:57.901785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb223d27-5381-4a27-b7d3-eaeff34dc683 service placement] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:42:57.906742 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0bfbd-b416-46e0-91ed-109c7a6492f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:42:57.906742 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0bfbd-b416-46e0-91ed-109c7a6492f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:42:57.907039 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0bfbd-b416-46e0-91ed-109c7a6492f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:42:57.907314 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0bfbd-b416-46e0-91ed-109c7a6492f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:42:57.916130 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f0bfbd-b416-46e0-91ed-109c7a6492f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:42:57.917558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f0bfbd-b416-46e0-91ed-109c7a6492f3 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:42:57.926611 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1169/2305] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:42:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:43:07.835604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=40 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:43:07.922430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=6 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:43:48.668581 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c0f11-6c33-4edb-9da7-5decf06e6a86 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:43:48.688087 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c0f11-6c33-4edb-9da7-5decf06e6a86 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:43:48.691815 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18856320-7aea-4ad9-afb4-9a2db70253c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:43:48.692436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18856320-7aea-4ad9-afb4-9a2db70253c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:43:48.692721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18856320-7aea-4ad9-afb4-9a2db70253c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:43:48.692994 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18856320-7aea-4ad9-afb4-9a2db70253c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:43:48.710089 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18856320-7aea-4ad9-afb4-9a2db70253c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:43:48.711521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18856320-7aea-4ad9-afb4-9a2db70253c5 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:43:48.733770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1137/2306] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:43:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:43:54.690704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0bfbd-b416-46e0-91ed-109c7a6492f3 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:43:54.699941 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0bfbd-b416-46e0-91ed-109c7a6492f3 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:43:54.704811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f9bc0-20dd-4565-81cd-39f916930dff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:43:54.705426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f9bc0-20dd-4565-81cd-39f916930dff service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:43:54.705709 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f9bc0-20dd-4565-81cd-39f916930dff service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:43:54.706022 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f9bc0-20dd-4565-81cd-39f916930dff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:43:54.716813 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f9bc0-20dd-4565-81cd-39f916930dff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:43:54.718273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f9bc0-20dd-4565-81cd-39f916930dff service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:43:54.729309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1170/2307] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:43:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:43:58.730378 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=17 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:44:03.741799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18856320-7aea-4ad9-afb4-9a2db70253c5 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:44:03.753536 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18856320-7aea-4ad9-afb4-9a2db70253c5 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:44:03.759132 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f29c30-123c-455e-9030-0b604dde3f84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:44:03.759741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29c30-123c-455e-9030-0b604dde3f84 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:44:03.760022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29c30-123c-455e-9030-0b604dde3f84 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:44:03.760294 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29c30-123c-455e-9030-0b604dde3f84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:44:03.768138 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f29c30-123c-455e-9030-0b604dde3f84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:44:03.769742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f29c30-123c-455e-9030-0b604dde3f84 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:44:03.796590 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1138/2308] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:44:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:44:04.724534 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=6 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:44:12.292279 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f9bc0-20dd-4565-81cd-39f916930dff service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:44:12.304114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f9bc0-20dd-4565-81cd-39f916930dff service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:44:12.309859 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6b6c3-f66c-4c78-bc02-0f0ea53ba982 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:44:12.311192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6b6c3-f66c-4c78-bc02-0f0ea53ba982 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:44:12.311473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6b6c3-f66c-4c78-bc02-0f0ea53ba982 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:44:12.311744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6b6c3-f66c-4c78-bc02-0f0ea53ba982 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:44:12.320088 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6b6c3-f66c-4c78-bc02-0f0ea53ba982 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:44:12.321777 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6b6c3-f66c-4c78-bc02-0f0ea53ba982 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:44:12.349819 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1171/2309] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:44:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:44:13.793451 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=13 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:44:22.347108 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=13 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:44:58.140844 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f29c30-123c-455e-9030-0b604dde3f84 service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:44:58.150125 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f29c30-123c-455e-9030-0b604dde3f84 service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:44:58.153697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881c14e-e3cc-4246-842a-bc5fa5e3c9a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:44:58.154390 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881c14e-e3cc-4246-842a-bc5fa5e3c9a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:44:58.154676 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881c14e-e3cc-4246-842a-bc5fa5e3c9a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:44:58.154951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881c14e-e3cc-4246-842a-bc5fa5e3c9a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:44:58.163344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881c14e-e3cc-4246-842a-bc5fa5e3c9a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:44:58.164778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881c14e-e3cc-4246-842a-bc5fa5e3c9a6 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:44:58.192430 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1139/2310] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:44:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:44:58.239461 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6b6c3-f66c-4c78-bc02-0f0ea53ba982 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:44:58.249449 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6b6c3-f66c-4c78-bc02-0f0ea53ba982 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:44:58.253120 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04218f-648e-4b91-b6f1-c72ffaf0b6d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:44:58.253729 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04218f-648e-4b91-b6f1-c72ffaf0b6d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:44:58.254043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04218f-648e-4b91-b6f1-c72ffaf0b6d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:44:58.254318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04218f-648e-4b91-b6f1-c72ffaf0b6d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:44:58.263104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04218f-648e-4b91-b6f1-c72ffaf0b6d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:44:58.264522 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04218f-648e-4b91-b6f1-c72ffaf0b6d8 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:44:58.291145 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1172/2311] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:44:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:45:08.189824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=13 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:45:08.288455 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=13 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:45:55.311269 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881c14e-e3cc-4246-842a-bc5fa5e3c9a6 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:45:55.320513 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881c14e-e3cc-4246-842a-bc5fa5e3c9a6 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:45:55.324208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2780795-f472-4f76-9dff-54a08783f07e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:45:55.324826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2780795-f472-4f76-9dff-54a08783f07e service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:45:55.325642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2780795-f472-4f76-9dff-54a08783f07e service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:45:55.326040 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2780795-f472-4f76-9dff-54a08783f07e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:45:55.334058 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2780795-f472-4f76-9dff-54a08783f07e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:45:55.335504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2780795-f472-4f76-9dff-54a08783f07e service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:45:55.344770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1140/2312] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:45:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:45:58.730389 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04218f-648e-4b91-b6f1-c72ffaf0b6d8 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:45:58.742782 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04218f-648e-4b91-b6f1-c72ffaf0b6d8 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:45:58.748222 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ca82e-8806-482d-a8a1-9be1940146df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:45:58.748830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ca82e-8806-482d-a8a1-9be1940146df service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:45:58.749112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ca82e-8806-482d-a8a1-9be1940146df service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:45:58.749386 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ca82e-8806-482d-a8a1-9be1940146df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:45:58.757219 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ca82e-8806-482d-a8a1-9be1940146df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:45:58.758829 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ca82e-8806-482d-a8a1-9be1940146df service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:45:58.786643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1173/2313] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:45:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:46:05.340074 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=6 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:46:08.783986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=13 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:46:54.644955 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2780795-f472-4f76-9dff-54a08783f07e service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:46:54.646524 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a2780795-f472-4f76-9dff-54a08783f07e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:46:54.662630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2780795-f472-4f76-9dff-54a08783f07e service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:46:54.667819 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25b104-090c-443f-92a4-25a0f3e6cee4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:46:54.668431 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25b104-090c-443f-92a4-25a0f3e6cee4 service placement] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:46:54.668710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25b104-090c-443f-92a4-25a0f3e6cee4 service placement] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:46:54.668978 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25b104-090c-443f-92a4-25a0f3e6cee4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:46:54.677427 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25b104-090c-443f-92a4-25a0f3e6cee4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:46:54.678999 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25b104-090c-443f-92a4-25a0f3e6cee4 service placement] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:46:54.689214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1141/2314] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:46:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:46:58.675382 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ca82e-8806-482d-a8a1-9be1940146df service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:46:58.676842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-451ca82e-8806-482d-a8a1-9be1940146df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:46:58.719222 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ca82e-8806-482d-a8a1-9be1940146df service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:46:58.726551 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02105f0a-9df6-4711-8fbc-31f229d6d1b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:46:58.727187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02105f0a-9df6-4711-8fbc-31f229d6d1b9 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:46:58.727472 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02105f0a-9df6-4711-8fbc-31f229d6d1b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:46:58.727748 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02105f0a-9df6-4711-8fbc-31f229d6d1b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:46:58.736826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02105f0a-9df6-4711-8fbc-31f229d6d1b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:46:58.738506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02105f0a-9df6-4711-8fbc-31f229d6d1b9 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:46:58.767904 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1174/2315] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:46:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:47:04.684553 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=6 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:47:08.764265 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=18 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:47:20.651652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25b104-090c-443f-92a4-25a0f3e6cee4 service placement] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:47:20.665650 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25b104-090c-443f-92a4-25a0f3e6cee4 service placement] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:47:20.671321 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d34dc-b077-462c-8670-748bbd4bebb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:47:20.671953 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d34dc-b077-462c-8670-748bbd4bebb4 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:47:20.672237 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d34dc-b077-462c-8670-748bbd4bebb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:47:20.672709 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d34dc-b077-462c-8670-748bbd4bebb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:47:20.674796 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2b4d34dc-b077-462c-8670-748bbd4bebb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:47:20.688551 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4d34dc-b077-462c-8670-748bbd4bebb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:47:20.690338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4d34dc-b077-462c-8670-748bbd4bebb4 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:47:20.701716 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1142/2316] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:47:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:47:30.697719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=6 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:48:59.084268 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02105f0a-9df6-4711-8fbc-31f229d6d1b9 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:48:59.085842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-02105f0a-9df6-4711-8fbc-31f229d6d1b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:48:59.162717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02105f0a-9df6-4711-8fbc-31f229d6d1b9 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:48:59.166791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37bcb4-b340-4d84-ab9e-b0641fb953ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:48:59.167414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37bcb4-b340-4d84-ab9e-b0641fb953ef service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:48:59.168195 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37bcb4-b340-4d84-ab9e-b0641fb953ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:48:59.168476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37bcb4-b340-4d84-ab9e-b0641fb953ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:48:59.170412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fa37bcb4-b340-4d84-ab9e-b0641fb953ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:48:59.246616 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37bcb4-b340-4d84-ab9e-b0641fb953ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:48:59.248138 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37bcb4-b340-4d84-ab9e-b0641fb953ef service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:48:59.260446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1175/2317] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:48:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:48:59.306732 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d34dc-b077-462c-8670-748bbd4bebb4 service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:48:59.316440 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d34dc-b077-462c-8670-748bbd4bebb4 service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:48:59.320224 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665178fb-f2eb-4e91-9ca6-8edb7d4bbc71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:48:59.320846 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665178fb-f2eb-4e91-9ca6-8edb7d4bbc71 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:48:59.321145 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665178fb-f2eb-4e91-9ca6-8edb7d4bbc71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:48:59.321432 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665178fb-f2eb-4e91-9ca6-8edb7d4bbc71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:48:59.331413 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665178fb-f2eb-4e91-9ca6-8edb7d4bbc71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:48:59.332856 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665178fb-f2eb-4e91-9ca6-8edb7d4bbc71 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:48:59.361640 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1143/2318] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:48:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:48:59.411720 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37bcb4-b340-4d84-ab9e-b0641fb953ef service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:48:59.420867 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37bcb4-b340-4d84-ab9e-b0641fb953ef service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:48:59.424989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51538221-170d-452d-b18d-52e16df68119 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:48:59.425707 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51538221-170d-452d-b18d-52e16df68119 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:48:59.426117 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51538221-170d-452d-b18d-52e16df68119 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:48:59.426459 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51538221-170d-452d-b18d-52e16df68119 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:48:59.428487 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-51538221-170d-452d-b18d-52e16df68119 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:48:59.517250 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51538221-170d-452d-b18d-52e16df68119 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:48:59.518652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51538221-170d-452d-b18d-52e16df68119 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:48:59.535121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:48:59.539639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1176/2319] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:48:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:48:59.658689 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665178fb-f2eb-4e91-9ca6-8edb7d4bbc71 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:48:59.668111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665178fb-f2eb-4e91-9ca6-8edb7d4bbc71 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:48:59.671531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b2d8f-151e-4895-b6a8-44cb747425f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:48:59.672697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b2d8f-151e-4895-b6a8-44cb747425f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:48:59.673005 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b2d8f-151e-4895-b6a8-44cb747425f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:48:59.673306 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b2d8f-151e-4895-b6a8-44cb747425f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:48:59.675268 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d46b2d8f-151e-4895-b6a8-44cb747425f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:48:59.690206 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b2d8f-151e-4895-b6a8-44cb747425f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:48:59.691644 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b2d8f-151e-4895-b6a8-44cb747425f0 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:48:59.704722 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1144/2320] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:48:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:49:09.698784 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=19 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:49:19.565996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51538221-170d-452d-b18d-52e16df68119 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:49:19.568123 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-51538221-170d-452d-b18d-52e16df68119 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:49:19.579911 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51538221-170d-452d-b18d-52e16df68119 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:49:19.584378 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73b3e8-a324-4707-993f-78a738e34fd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:49:19.585000 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73b3e8-a324-4707-993f-78a738e34fd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:49:19.585281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73b3e8-a324-4707-993f-78a738e34fd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:49:19.585554 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73b3e8-a324-4707-993f-78a738e34fd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:49:19.587538 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8b73b3e8-a324-4707-993f-78a738e34fd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:49:19.663799 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b73b3e8-a324-4707-993f-78a738e34fd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:49:19.665462 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b73b3e8-a324-4707-993f-78a738e34fd5 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:49:19.676067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1177/2321] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:49:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:49:19.739025 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b2d8f-151e-4895-b6a8-44cb747425f0 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:49:19.748430 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b2d8f-151e-4895-b6a8-44cb747425f0 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:49:19.752803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9f634-ea28-4b56-85c5-f16e28606fbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:49:19.753426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9f634-ea28-4b56-85c5-f16e28606fbb service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:49:19.753711 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9f634-ea28-4b56-85c5-f16e28606fbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:49:19.754019 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9f634-ea28-4b56-85c5-f16e28606fbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:49:19.755930 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5df9f634-ea28-4b56-85c5-f16e28606fbb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:49:19.838250 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df9f634-ea28-4b56-85c5-f16e28606fbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:49:19.839840 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df9f634-ea28-4b56-85c5-f16e28606fbb service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:49:19.850543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1145/2322] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:49:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:49:29.670579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=33 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:49:29.845783 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=33 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:49:58.518469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73b3e8-a324-4707-993f-78a738e34fd5 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:49:58.520074 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8b73b3e8-a324-4707-993f-78a738e34fd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:49:58.535855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73b3e8-a324-4707-993f-78a738e34fd5 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:49:58.541697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4c5e0-54eb-4a55-b4d6-ab224ba23d5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:49:58.542351 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4c5e0-54eb-4a55-b4d6-ab224ba23d5a service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:49:58.542631 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4c5e0-54eb-4a55-b4d6-ab224ba23d5a service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:49:58.542915 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4c5e0-54eb-4a55-b4d6-ab224ba23d5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:49:58.544735 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2ef4c5e0-54eb-4a55-b4d6-ab224ba23d5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:49:58.617791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4c5e0-54eb-4a55-b4d6-ab224ba23d5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:49:58.620549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4c5e0-54eb-4a55-b4d6-ab224ba23d5a service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:49:58.632044 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1178/2323] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:49:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:49:58.798137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9f634-ea28-4b56-85c5-f16e28606fbb service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:49:58.810671 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9f634-ea28-4b56-85c5-f16e28606fbb service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:49:58.816447 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c858f2-ac7a-4eba-85e1-dcbd9c0a4ca3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:49:58.817060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c858f2-ac7a-4eba-85e1-dcbd9c0a4ca3 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:49:58.817345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c858f2-ac7a-4eba-85e1-dcbd9c0a4ca3 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:49:58.817647 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c858f2-ac7a-4eba-85e1-dcbd9c0a4ca3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:49:58.819551 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b9c858f2-ac7a-4eba-85e1-dcbd9c0a4ca3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:49:58.890687 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c858f2-ac7a-4eba-85e1-dcbd9c0a4ca3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:49:58.892887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c858f2-ac7a-4eba-85e1-dcbd9c0a4ca3 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:49:58.902555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1146/2324] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:49:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:50:08.627601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=33 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:50:08.898003 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=33 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:50:58.303217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4c5e0-54eb-4a55-b4d6-ab224ba23d5a service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:50:58.304890 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2ef4c5e0-54eb-4a55-b4d6-ab224ba23d5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:50:58.375470 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4c5e0-54eb-4a55-b4d6-ab224ba23d5a service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:50:58.379135 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a5797-4ad8-4a39-b7bc-e4b341df1595 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:50:58.379752 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a5797-4ad8-4a39-b7bc-e4b341df1595 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:50:58.380037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a5797-4ad8-4a39-b7bc-e4b341df1595 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:50:58.380311 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a5797-4ad8-4a39-b7bc-e4b341df1595 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:50:58.382233 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-788a5797-4ad8-4a39-b7bc-e4b341df1595 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:50:58.392035 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788a5797-4ad8-4a39-b7bc-e4b341df1595 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:50:58.393459 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788a5797-4ad8-4a39-b7bc-e4b341df1595 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:50:58.401412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1179/2325] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:50:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:50:59.977396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c858f2-ac7a-4eba-85e1-dcbd9c0a4ca3 service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:50:59.989179 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c858f2-ac7a-4eba-85e1-dcbd9c0a4ca3 service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:50:59.993180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ce22b-9a6b-463d-81e0-711698a28e33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:50:59.993811 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ce22b-9a6b-463d-81e0-711698a28e33 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:50:59.994120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ce22b-9a6b-463d-81e0-711698a28e33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:50:59.994398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ce22b-9a6b-463d-81e0-711698a28e33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:50:59.996462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-426ce22b-9a6b-463d-81e0-711698a28e33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:51:00.073729 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ce22b-9a6b-463d-81e0-711698a28e33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:51:00.075249 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ce22b-9a6b-463d-81e0-711698a28e33 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:51:00.085700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1147/2326] 10.222.0.83 () {60 vars in 1396 bytes} [Mon Aug 1 15:50:59 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:51:02.001665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a5797-4ad8-4a39-b7bc-e4b341df1595 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:51:02.016038 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a5797-4ad8-4a39-b7bc-e4b341df1595 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:51:02.021729 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61be4e5-0510-4d9a-8dac-fe32d1e428a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:51:02.022387 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61be4e5-0510-4d9a-8dac-fe32d1e428a8 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:51:02.022671 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61be4e5-0510-4d9a-8dac-fe32d1e428a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:51:02.022950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61be4e5-0510-4d9a-8dac-fe32d1e428a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:51:02.024797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c61be4e5-0510-4d9a-8dac-fe32d1e428a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:51:02.039011 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61be4e5-0510-4d9a-8dac-fe32d1e428a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:51:02.040644 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61be4e5-0510-4d9a-8dac-fe32d1e428a8 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:51:02.052802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1180/2327] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:51:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:51:10.080773 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=33 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:51:12.047768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=37 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:51:22.881824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4e562-15ca-4222-85c8-16f858b996f0 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:51:22.882337 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4e562-15ca-4222-85c8-16f858b996f0 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:51:22.882445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4e562-15ca-4222-85c8-16f858b996f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:51:22.948164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-91c4e562-15ca-4222-85c8-16f858b996f0 None None] MFA Rules not processed for user `f36c6987ef774a1684c55f885e2a4f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:51:22.949411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-91c4e562-15ca-4222-85c8-16f858b996f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:51:23.021642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1148/2328] 10.222.0.83 () {58 vars in 967 bytes} [Mon Aug 1 15:51:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:51:23.086791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61be4e5-0510-4d9a-8dac-fe32d1e428a8 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:51:23.088443 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c61be4e5-0510-4d9a-8dac-fe32d1e428a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:51:23.190316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61be4e5-0510-4d9a-8dac-fe32d1e428a8 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:51:23.195474 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe106a8c-7391-451f-a853-7fac34ceff93 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6873a3e440bd4b63b0082fc85beef49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:51:23.196095 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe106a8c-7391-451f-a853-7fac34ceff93 service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:51:23.196378 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe106a8c-7391-451f-a853-7fac34ceff93 service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:51:23.196654 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe106a8c-7391-451f-a853-7fac34ceff93 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:51:23.205360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe106a8c-7391-451f-a853-7fac34ceff93 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:51:23.207036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe106a8c-7391-451f-a853-7fac34ceff93 service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:51:23.215378 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1181/2329] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Aug 1 15:51:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:51:23.605220 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4e562-15ca-4222-85c8-16f858b996f0 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:51:23.606825 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-91c4e562-15ca-4222-85c8-16f858b996f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:51:23.619275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4e562-15ca-4222-85c8-16f858b996f0 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:51:23.623053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ea12b-b7b2-4523-a9bb-4b73936295bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:51:23.623627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ea12b-b7b2-4523-a9bb-4b73936295bf service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:51:23.623910 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ea12b-b7b2-4523-a9bb-4b73936295bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:51:23.624274 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ea12b-b7b2-4523-a9bb-4b73936295bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:51:23.625979 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bc9ea12b-b7b2-4523-a9bb-4b73936295bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:51:23.710384 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ea12b-b7b2-4523-a9bb-4b73936295bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:51:23.711857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ea12b-b7b2-4523-a9bb-4b73936295bf service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:51:23.732278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1149/2330] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:51:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:51:27.143603 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe106a8c-7391-451f-a853-7fac34ceff93 service swift] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:51:27.155831 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe106a8c-7391-451f-a853-7fac34ceff93 service swift] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:51:27.161147 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba640e-0a30-420b-ab4f-e372ffeea9bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:51:27.161760 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba640e-0a30-420b-ab4f-e372ffeea9bc service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:51:27.162083 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba640e-0a30-420b-ab4f-e372ffeea9bc service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:51:27.162350 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba640e-0a30-420b-ab4f-e372ffeea9bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:51:27.164144 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7cba640e-0a30-420b-ab4f-e372ffeea9bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:51:27.176461 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba640e-0a30-420b-ab4f-e372ffeea9bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:51:27.178961 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba640e-0a30-420b-ab4f-e372ffeea9bc service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:51:27.191795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1182/2331] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:51:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:51:30.995950 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ea12b-b7b2-4523-a9bb-4b73936295bf service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:51:30.997386 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bc9ea12b-b7b2-4523-a9bb-4b73936295bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:51:31.009561 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ea12b-b7b2-4523-a9bb-4b73936295bf service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:51:31.014516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d4917-a33f-4870-b8bb-63aa03e38540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:51:31.015172 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d4917-a33f-4870-b8bb-63aa03e38540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:51:31.015456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d4917-a33f-4870-b8bb-63aa03e38540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:51:31.015734 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d4917-a33f-4870-b8bb-63aa03e38540 admin admin] PATH_INFO: `/v3/users/a26637889378427eb884eed624606fd5` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:51:31.030590 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d4917-a33f-4870-b8bb-63aa03e38540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a26637889378427eb884eed624606fd5)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:51:31.032546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d4917-a33f-4870-b8bb-63aa03e38540 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:51:31.044485 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:51:31.091400 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-565d4917-a33f-4870-b8bb-63aa03e38540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a26637889378427eb884eed624606fd5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:51:31.093232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-565d4917-a33f-4870-b8bb-63aa03e38540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a26637889378427eb884eed624606fd5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:51:31.102793 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-565d4917-a33f-4870-b8bb-63aa03e38540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a26637889378427eb884eed624606fd5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:51:31.116063 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-565d4917-a33f-4870-b8bb-63aa03e38540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a26637889378427eb884eed624606fd5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:51:31.116755 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-565d4917-a33f-4870-b8bb-63aa03e38540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a26637889378427eb884eed624606fd5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:51:31.134944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1150/2332] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:51:30 2022] DELETE /identity/v3/users/a26637889378427eb884eed624606fd5 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:51:31.327455 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba640e-0a30-420b-ab4f-e372ffeea9bc service placement] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:51:31.339734 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba640e-0a30-420b-ab4f-e372ffeea9bc service placement] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:51:31.400802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80cb9b-a653-4ebd-b6ef-23badb2cf454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:51:31.401453 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80cb9b-a653-4ebd-b6ef-23badb2cf454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:51:31.401733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80cb9b-a653-4ebd-b6ef-23badb2cf454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:51:31.402045 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80cb9b-a653-4ebd-b6ef-23badb2cf454 admin admin] PATH_INFO: `/v3/projects/3762799522514a268a4c8b8069baa882` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:51:31.403245 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80cb9b-a653-4ebd-b6ef-23badb2cf454 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3762799522514a268a4c8b8069baa882)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:51:31.405711 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80cb9b-a653-4ebd-b6ef-23badb2cf454 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:51:31.453995 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9a80cb9b-a653-4ebd-b6ef-23badb2cf454 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3762799522514a268a4c8b8069baa882', 'request_id': 'req-9a80cb9b-a653-4ebd-b6ef-23badb2cf454'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:51:31.459635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9a80cb9b-a653-4ebd-b6ef-23badb2cf454 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3762799522514a268a4c8b8069baa882', 'request_id': 'req-9a80cb9b-a653-4ebd-b6ef-23badb2cf454'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:51:31.460786 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:51:31.468833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9a80cb9b-a653-4ebd-b6ef-23badb2cf454 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3762799522514a268a4c8b8069baa882', 'request_id': 'req-9a80cb9b-a653-4ebd-b6ef-23badb2cf454'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:51:31.482461 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9a80cb9b-a653-4ebd-b6ef-23badb2cf454 admin admin] The token cache is being invalidate because project 3762799522514a268a4c8b8069baa882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:51:31.482750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9a80cb9b-a653-4ebd-b6ef-23badb2cf454 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:51:31.483969 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9a80cb9b-a653-4ebd-b6ef-23badb2cf454 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:51:31.485321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1183/2333] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:51:31 2022] DELETE /identity/v3/projects/3762799522514a268a4c8b8069baa882 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:51:31.489120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d4917-a33f-4870-b8bb-63aa03e38540 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:51:31.490569 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-565d4917-a33f-4870-b8bb-63aa03e38540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:51:31.507416 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d4917-a33f-4870-b8bb-63aa03e38540 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:51:31.576516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fed99b-e0c9-46ec-9681-f331bcc6b2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:51:31.577833 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fed99b-e0c9-46ec-9681-f331bcc6b2a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:51:31.578210 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fed99b-e0c9-46ec-9681-f331bcc6b2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:51:31.578550 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fed99b-e0c9-46ec-9681-f331bcc6b2a7 admin admin] PATH_INFO: `/v3/users/567966b846814025b553ed2d00f9db6f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:51:31.598236 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fed99b-e0c9-46ec-9681-f331bcc6b2a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=567966b846814025b553ed2d00f9db6f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:51:31.600830 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fed99b-e0c9-46ec-9681-f331bcc6b2a7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:51:31.652804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-77fed99b-e0c9-46ec-9681-f331bcc6b2a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '567966b846814025b553ed2d00f9db6f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:51:31.653507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-77fed99b-e0c9-46ec-9681-f331bcc6b2a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '567966b846814025b553ed2d00f9db6f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:51:31.662222 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-77fed99b-e0c9-46ec-9681-f331bcc6b2a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '567966b846814025b553ed2d00f9db6f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:51:31.674841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-77fed99b-e0c9-46ec-9681-f331bcc6b2a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '567966b846814025b553ed2d00f9db6f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:51:31.675568 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-77fed99b-e0c9-46ec-9681-f331bcc6b2a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '567966b846814025b553ed2d00f9db6f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:51:31.684278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:51:31.691798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1151/2334] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:51:31 2022] DELETE /identity/v3/users/567966b846814025b553ed2d00f9db6f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:51:31.880683 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80cb9b-a653-4ebd-b6ef-23badb2cf454 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:51:31.882589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9a80cb9b-a653-4ebd-b6ef-23badb2cf454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:51:31.902296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80cb9b-a653-4ebd-b6ef-23badb2cf454 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:51:31.964715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403c269-36a4-4e9a-bf81-6e922eb75f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:51:31.965376 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403c269-36a4-4e9a-bf81-6e922eb75f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:51:31.965665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403c269-36a4-4e9a-bf81-6e922eb75f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:51:31.965969 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403c269-36a4-4e9a-bf81-6e922eb75f4d admin admin] PATH_INFO: `/v3/projects/5fcf85e8b32f40f888f4bde52b7e1a8e` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:51:31.977262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b403c269-36a4-4e9a-bf81-6e922eb75f4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fcf85e8b32f40f888f4bde52b7e1a8e)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:51:31.979245 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b403c269-36a4-4e9a-bf81-6e922eb75f4d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:51:32.031567 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b403c269-36a4-4e9a-bf81-6e922eb75f4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fcf85e8b32f40f888f4bde52b7e1a8e', 'request_id': 'req-b403c269-36a4-4e9a-bf81-6e922eb75f4d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:51:32.038335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b403c269-36a4-4e9a-bf81-6e922eb75f4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fcf85e8b32f40f888f4bde52b7e1a8e', 'request_id': 'req-b403c269-36a4-4e9a-bf81-6e922eb75f4d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:51:32.047539 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b403c269-36a4-4e9a-bf81-6e922eb75f4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fcf85e8b32f40f888f4bde52b7e1a8e', 'request_id': 'req-b403c269-36a4-4e9a-bf81-6e922eb75f4d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:51:32.067571 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b403c269-36a4-4e9a-bf81-6e922eb75f4d admin admin] The token cache is being invalidate because project 5fcf85e8b32f40f888f4bde52b7e1a8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:51:32.067870 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b403c269-36a4-4e9a-bf81-6e922eb75f4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:51:32.068530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b403c269-36a4-4e9a-bf81-6e922eb75f4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:51:32.070084 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1184/2335] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:51:31 2022] DELETE /identity/v3/projects/5fcf85e8b32f40f888f4bde52b7e1a8e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:51:41.695077 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:51:42.060296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=59,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:51:58.570852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fed99b-e0c9-46ec-9681-f331bcc6b2a7 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:51:58.572493 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-77fed99b-e0c9-46ec-9681-f331bcc6b2a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:51:58.678115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fed99b-e0c9-46ec-9681-f331bcc6b2a7 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:51:58.751084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ecb31-3db7-43e4-937c-b36f95d6d4aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:51:58.751723 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ecb31-3db7-43e4-937c-b36f95d6d4aa service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:51:58.752006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ecb31-3db7-43e4-937c-b36f95d6d4aa service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:51:58.752286 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ecb31-3db7-43e4-937c-b36f95d6d4aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:51:58.754207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9b3ecb31-3db7-43e4-937c-b36f95d6d4aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:51:58.824879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:51:58.828327 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ecb31-3db7-43e4-937c-b36f95d6d4aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:51:58.830557 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ecb31-3db7-43e4-937c-b36f95d6d4aa service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:51:58.839858 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1152/2336] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:51:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:52:00.649866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403c269-36a4-4e9a-bf81-6e922eb75f4d admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:52:00.651299 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b403c269-36a4-4e9a-bf81-6e922eb75f4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:52:00.764013 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403c269-36a4-4e9a-bf81-6e922eb75f4d admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:52:00.769823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc1f36-2ec3-4e65-9962-f0bd4e7e3b5e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:52:00.770481 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc1f36-2ec3-4e65-9962-f0bd4e7e3b5e service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:52:00.770768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc1f36-2ec3-4e65-9962-f0bd4e7e3b5e service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:52:00.771046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc1f36-2ec3-4e65-9962-f0bd4e7e3b5e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:52:00.772810 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-18cc1f36-2ec3-4e65-9962-f0bd4e7e3b5e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:52:00.784186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc1f36-2ec3-4e65-9962-f0bd4e7e3b5e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:52:00.785774 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc1f36-2ec3-4e65-9962-f0bd4e7e3b5e service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:52:00.795543 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1185/2337] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:52:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:52:08.835536 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:52:10.791487 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=50 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:53:21.733157 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ecb31-3db7-43e4-937c-b36f95d6d4aa service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:53:21.735758 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9b3ecb31-3db7-43e4-937c-b36f95d6d4aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:53:21.814590 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ecb31-3db7-43e4-937c-b36f95d6d4aa service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:53:21.818917 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec163d3-211d-4d5c-a783-a6916c804c6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:53:21.819548 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec163d3-211d-4d5c-a783-a6916c804c6a service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:53:21.819831 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec163d3-211d-4d5c-a783-a6916c804c6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:53:21.820111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec163d3-211d-4d5c-a783-a6916c804c6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:53:21.821931 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-cec163d3-211d-4d5c-a783-a6916c804c6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:53:21.887443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec163d3-211d-4d5c-a783-a6916c804c6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:53:21.888916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec163d3-211d-4d5c-a783-a6916c804c6a service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:53:21.899186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1153/2338] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:53:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:53:25.360500 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc1f36-2ec3-4e65-9962-f0bd4e7e3b5e service placement] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:53:25.382082 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc1f36-2ec3-4e65-9962-f0bd4e7e3b5e service placement] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:53:25.387522 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b25ae-0d61-485a-a1a1-6c61f87ffe6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:53:25.388274 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b25ae-0d61-485a-a1a1-6c61f87ffe6f service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:53:25.388560 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b25ae-0d61-485a-a1a1-6c61f87ffe6f service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:53:25.388829 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b25ae-0d61-485a-a1a1-6c61f87ffe6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:53:25.391102 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-977b25ae-0d61-485a-a1a1-6c61f87ffe6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:53:25.404400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b25ae-0d61-485a-a1a1-6c61f87ffe6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:53:25.406065 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b25ae-0d61-485a-a1a1-6c61f87ffe6f service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:53:25.418625 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1186/2339] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:53:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:53:29.091975 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec163d3-211d-4d5c-a783-a6916c804c6a service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:53:29.093495 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-cec163d3-211d-4d5c-a783-a6916c804c6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:53:29.108376 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec163d3-211d-4d5c-a783-a6916c804c6a service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:53:29.113014 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51750e-f601-4265-b2c2-c957bad508f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:53:29.113651 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51750e-f601-4265-b2c2-c957bad508f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:53:29.113963 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51750e-f601-4265-b2c2-c957bad508f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:53:29.114250 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51750e-f601-4265-b2c2-c957bad508f9 admin admin] PATH_INFO: `/v3/users/6d903b4429664fceaa1344f1b99791dd` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:53:29.116055 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de51750e-f601-4265-b2c2-c957bad508f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d903b4429664fceaa1344f1b99791dd)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:53:29.118006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de51750e-f601-4265-b2c2-c957bad508f9 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:53:29.172094 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-de51750e-f601-4265-b2c2-c957bad508f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d903b4429664fceaa1344f1b99791dd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:53:29.172793 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-de51750e-f601-4265-b2c2-c957bad508f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d903b4429664fceaa1344f1b99791dd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:53:29.182391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-de51750e-f601-4265-b2c2-c957bad508f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d903b4429664fceaa1344f1b99791dd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:53:29.201316 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-de51750e-f601-4265-b2c2-c957bad508f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d903b4429664fceaa1344f1b99791dd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:53:29.202093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-de51750e-f601-4265-b2c2-c957bad508f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d903b4429664fceaa1344f1b99791dd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:53:29.215417 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1154/2340] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:53:29 2022] DELETE /identity/v3/users/6d903b4429664fceaa1344f1b99791dd => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:53:29.399116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b25ae-0d61-485a-a1a1-6c61f87ffe6f service placement] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:53:29.411319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b25ae-0d61-485a-a1a1-6c61f87ffe6f service placement] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:53:29.468375 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b9810-9708-4e66-a77b-2677e997911b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:53:29.469019 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b9810-9708-4e66-a77b-2677e997911b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:53:29.469303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b9810-9708-4e66-a77b-2677e997911b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:53:29.469576 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b9810-9708-4e66-a77b-2677e997911b admin admin] PATH_INFO: `/v3/projects/63125418cd9949cdbf76099952b31950` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:53:29.470777 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b9810-9708-4e66-a77b-2677e997911b admin admin] RBAC: Authorizing `identity:delete_project(project_id=63125418cd9949cdbf76099952b31950)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:53:29.472934 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b9810-9708-4e66-a77b-2677e997911b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:53:29.522751 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-cf6b9810-9708-4e66-a77b-2677e997911b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63125418cd9949cdbf76099952b31950', 'request_id': 'req-cf6b9810-9708-4e66-a77b-2677e997911b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:53:29.528500 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-cf6b9810-9708-4e66-a77b-2677e997911b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63125418cd9949cdbf76099952b31950', 'request_id': 'req-cf6b9810-9708-4e66-a77b-2677e997911b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:53:29.534684 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-cf6b9810-9708-4e66-a77b-2677e997911b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63125418cd9949cdbf76099952b31950', 'request_id': 'req-cf6b9810-9708-4e66-a77b-2677e997911b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:53:29.549002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-cf6b9810-9708-4e66-a77b-2677e997911b admin admin] The token cache is being invalidate because project 63125418cd9949cdbf76099952b31950 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:53:29.549367 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-cf6b9810-9708-4e66-a77b-2677e997911b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:53:29.549916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-cf6b9810-9708-4e66-a77b-2677e997911b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:53:29.551462 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1187/2341] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:53:29 2022] DELETE /identity/v3/projects/63125418cd9949cdbf76099952b31950 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:53:29.555561 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51750e-f601-4265-b2c2-c957bad508f9 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:53:29.556991 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-de51750e-f601-4265-b2c2-c957bad508f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:53:29.565179 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:53:29.571819 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51750e-f601-4265-b2c2-c957bad508f9 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:53:29.628515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86d79c-8c6f-431f-a53f-6caefc4ec0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:53:29.629164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86d79c-8c6f-431f-a53f-6caefc4ec0bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:53:29.629452 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86d79c-8c6f-431f-a53f-6caefc4ec0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:53:29.629730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86d79c-8c6f-431f-a53f-6caefc4ec0bd admin admin] PATH_INFO: `/v3/users/23e07f301bb848188defaddfa9db59b0` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:53:29.643527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86d79c-8c6f-431f-a53f-6caefc4ec0bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=23e07f301bb848188defaddfa9db59b0)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:53:29.645484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86d79c-8c6f-431f-a53f-6caefc4ec0bd admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:53:29.691415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-dd86d79c-8c6f-431f-a53f-6caefc4ec0bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23e07f301bb848188defaddfa9db59b0'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:53:29.691984 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-dd86d79c-8c6f-431f-a53f-6caefc4ec0bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23e07f301bb848188defaddfa9db59b0'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:53:29.699835 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-dd86d79c-8c6f-431f-a53f-6caefc4ec0bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23e07f301bb848188defaddfa9db59b0'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:53:29.712489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-dd86d79c-8c6f-431f-a53f-6caefc4ec0bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23e07f301bb848188defaddfa9db59b0'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:53:29.713155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-dd86d79c-8c6f-431f-a53f-6caefc4ec0bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23e07f301bb848188defaddfa9db59b0'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:53:29.726004 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1155/2342] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:53:29 2022] DELETE /identity/v3/users/23e07f301bb848188defaddfa9db59b0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:53:29.901998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b9810-9708-4e66-a77b-2677e997911b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:53:29.903819 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-cf6b9810-9708-4e66-a77b-2677e997911b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:53:29.919282 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b9810-9708-4e66-a77b-2677e997911b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:53:29.984623 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09c153-8d54-4902-9159-d8ce7ca33be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:53:29.985187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09c153-8d54-4902-9159-d8ce7ca33be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:53:29.985465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09c153-8d54-4902-9159-d8ce7ca33be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:53:29.985737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09c153-8d54-4902-9159-d8ce7ca33be5 admin admin] PATH_INFO: `/v3/projects/935097df97b74d04aa0603250d968f5c` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:53:29.994458 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:53:29.998190 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09c153-8d54-4902-9159-d8ce7ca33be5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=935097df97b74d04aa0603250d968f5c)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:53:30.000339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09c153-8d54-4902-9159-d8ce7ca33be5 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:53:30.048463 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fe09c153-8d54-4902-9159-d8ce7ca33be5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '935097df97b74d04aa0603250d968f5c', 'request_id': 'req-fe09c153-8d54-4902-9159-d8ce7ca33be5'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:53:30.054332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fe09c153-8d54-4902-9159-d8ce7ca33be5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '935097df97b74d04aa0603250d968f5c', 'request_id': 'req-fe09c153-8d54-4902-9159-d8ce7ca33be5'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:53:30.060629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fe09c153-8d54-4902-9159-d8ce7ca33be5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '935097df97b74d04aa0603250d968f5c', 'request_id': 'req-fe09c153-8d54-4902-9159-d8ce7ca33be5'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:53:30.073826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fe09c153-8d54-4902-9159-d8ce7ca33be5 admin admin] The token cache is being invalidate because project 935097df97b74d04aa0603250d968f5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:53:30.074142 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fe09c153-8d54-4902-9159-d8ce7ca33be5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:53:30.074769 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fe09c153-8d54-4902-9159-d8ce7ca33be5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:53:30.076274 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1188/2343] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:53:29 2022] DELETE /identity/v3/projects/935097df97b74d04aa0603250d968f5c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:53:39.721680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:53:40.070495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=25,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:53:59.051097 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86d79c-8c6f-431f-a53f-6caefc4ec0bd admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:53:59.053381 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-dd86d79c-8c6f-431f-a53f-6caefc4ec0bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:53:59.125008 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86d79c-8c6f-431f-a53f-6caefc4ec0bd admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:53:59.178643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483801b6-e131-44d7-af3c-d46d8b4f0dd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:53:59.179275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483801b6-e131-44d7-af3c-d46d8b4f0dd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:53:59.179588 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483801b6-e131-44d7-af3c-d46d8b4f0dd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:53:59.179872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483801b6-e131-44d7-af3c-d46d8b4f0dd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:53:59.181652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-483801b6-e131-44d7-af3c-d46d8b4f0dd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:53:59.258179 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483801b6-e131-44d7-af3c-d46d8b4f0dd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:53:59.259563 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483801b6-e131-44d7-af3c-d46d8b4f0dd4 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:53:59.280479 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1156/2344] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:53:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:54:00.221240 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09c153-8d54-4902-9159-d8ce7ca33be5 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:54:00.222946 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fe09c153-8d54-4902-9159-d8ce7ca33be5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:54:00.235543 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09c153-8d54-4902-9159-d8ce7ca33be5 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:54:00.239119 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c13d9-bdc8-4a94-a08b-069490538aa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:54:00.239750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c13d9-bdc8-4a94-a08b-069490538aa6 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:54:00.240034 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c13d9-bdc8-4a94-a08b-069490538aa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:54:00.240308 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c13d9-bdc8-4a94-a08b-069490538aa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:54:00.242080 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-146c13d9-bdc8-4a94-a08b-069490538aa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:54:00.255735 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c13d9-bdc8-4a94-a08b-069490538aa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:54:00.257154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c13d9-bdc8-4a94-a08b-069490538aa6 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:54:00.267942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1189/2345] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:54:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:54:09.277849 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=85 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:54:10.263237 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=6 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:54:19.361452 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483801b6-e131-44d7-af3c-d46d8b4f0dd4 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:54:19.363040 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-483801b6-e131-44d7-af3c-d46d8b4f0dd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:54:19.376691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483801b6-e131-44d7-af3c-d46d8b4f0dd4 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:54:19.382927 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded80b9-d8bc-4359-abe5-46be433b9fe6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:54:19.383553 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded80b9-d8bc-4359-abe5-46be433b9fe6 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:54:19.383837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded80b9-d8bc-4359-abe5-46be433b9fe6 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:54:19.384107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded80b9-d8bc-4359-abe5-46be433b9fe6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:54:19.386030 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3ded80b9-d8bc-4359-abe5-46be433b9fe6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:54:19.459087 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded80b9-d8bc-4359-abe5-46be433b9fe6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:54:19.460723 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded80b9-d8bc-4359-abe5-46be433b9fe6 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:54:19.480130 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1157/2346] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:54:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:54:20.531477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c13d9-bdc8-4a94-a08b-069490538aa6 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:54:20.533144 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-146c13d9-bdc8-4a94-a08b-069490538aa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:54:20.547896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c13d9-bdc8-4a94-a08b-069490538aa6 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:54:20.554049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51207e9e-9f80-47c3-96ab-de44a9d04376 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:54:20.554662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51207e9e-9f80-47c3-96ab-de44a9d04376 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:54:20.554943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51207e9e-9f80-47c3-96ab-de44a9d04376 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:54:20.555217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51207e9e-9f80-47c3-96ab-de44a9d04376 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:54:20.556942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-51207e9e-9f80-47c3-96ab-de44a9d04376 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:54:20.625263 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51207e9e-9f80-47c3-96ab-de44a9d04376 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:54:20.626904 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51207e9e-9f80-47c3-96ab-de44a9d04376 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:54:20.645559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1190/2347] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:54:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:54:29.477221 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=36 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:54:30.642971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=36 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:55:01.099428 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded80b9-d8bc-4359-abe5-46be433b9fe6 service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:55:01.100973 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3ded80b9-d8bc-4359-abe5-46be433b9fe6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:55:01.182935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded80b9-d8bc-4359-abe5-46be433b9fe6 service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:55:01.186623 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf8242-0b3c-43d5-b776-17b6371ea4f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:55:01.187238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf8242-0b3c-43d5-b776-17b6371ea4f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:55:01.187518 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf8242-0b3c-43d5-b776-17b6371ea4f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:55:01.187791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf8242-0b3c-43d5-b776-17b6371ea4f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:55:01.189554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-33cf8242-0b3c-43d5-b776-17b6371ea4f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:55:01.256703 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf8242-0b3c-43d5-b776-17b6371ea4f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:55:01.258213 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf8242-0b3c-43d5-b776-17b6371ea4f7 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:55:01.277362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1158/2348] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:55:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:55:01.364404 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51207e9e-9f80-47c3-96ab-de44a9d04376 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:55:01.375533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51207e9e-9f80-47c3-96ab-de44a9d04376 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:55:01.379745 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c128ec2-1b5d-41e1-9c8f-e3cb2f2366e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:55:01.380473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c128ec2-1b5d-41e1-9c8f-e3cb2f2366e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:55:01.380758 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c128ec2-1b5d-41e1-9c8f-e3cb2f2366e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:55:01.381039 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c128ec2-1b5d-41e1-9c8f-e3cb2f2366e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:55:01.383255 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9c128ec2-1b5d-41e1-9c8f-e3cb2f2366e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:55:01.462747 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c128ec2-1b5d-41e1-9c8f-e3cb2f2366e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:55:01.464188 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c128ec2-1b5d-41e1-9c8f-e3cb2f2366e5 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:55:01.483360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1191/2349] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:55:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:55:01.531997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf8242-0b3c-43d5-b776-17b6371ea4f7 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:55:01.540461 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf8242-0b3c-43d5-b776-17b6371ea4f7 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:55:01.544096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52105e9d-a1a7-4616-828c-d7540f2bf19d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:55:01.544701 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52105e9d-a1a7-4616-828c-d7540f2bf19d service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:55:01.544985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52105e9d-a1a7-4616-828c-d7540f2bf19d service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:55:01.545470 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52105e9d-a1a7-4616-828c-d7540f2bf19d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:55:01.547231 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-52105e9d-a1a7-4616-828c-d7540f2bf19d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:55:01.606450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52105e9d-a1a7-4616-828c-d7540f2bf19d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:55:01.607804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52105e9d-a1a7-4616-828c-d7540f2bf19d service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:55:01.616619 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1159/2350] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:55:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:55:11.480632 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=36 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:55:11.612626 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=88 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:56:02.306717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c128ec2-1b5d-41e1-9c8f-e3cb2f2366e5 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:56:02.316215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c128ec2-1b5d-41e1-9c8f-e3cb2f2366e5 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:56:02.320659 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211a232-05f8-4b53-9e12-4adfa3a57f9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:56:02.321546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211a232-05f8-4b53-9e12-4adfa3a57f9e service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:56:02.321834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211a232-05f8-4b53-9e12-4adfa3a57f9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:56:02.322139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211a232-05f8-4b53-9e12-4adfa3a57f9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:56:02.323968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c211a232-05f8-4b53-9e12-4adfa3a57f9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:56:02.335282 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211a232-05f8-4b53-9e12-4adfa3a57f9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:56:02.336811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211a232-05f8-4b53-9e12-4adfa3a57f9e service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:56:02.345945 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1192/2351] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:56:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:56:12.341849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=6 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:57:01.886833 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52105e9d-a1a7-4616-828c-d7540f2bf19d service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:57:01.899530 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52105e9d-a1a7-4616-828c-d7540f2bf19d service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:57:01.907223 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40a034-a037-4529-9362-dae13edff1e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:57:01.907875 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40a034-a037-4529-9362-dae13edff1e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:57:01.908165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40a034-a037-4529-9362-dae13edff1e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:57:01.908430 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40a034-a037-4529-9362-dae13edff1e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:57:01.910390 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-cf40a034-a037-4529-9362-dae13edff1e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:57:01.989610 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40a034-a037-4529-9362-dae13edff1e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:57:01.991215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40a034-a037-4529-9362-dae13edff1e1 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:57:02.026434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1160/2352] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:57:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:57:07.689169 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211a232-05f8-4b53-9e12-4adfa3a57f9e service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:57:07.690993 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c211a232-05f8-4b53-9e12-4adfa3a57f9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:57:07.814589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211a232-05f8-4b53-9e12-4adfa3a57f9e service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:57:07.820227 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e96e0-3ac2-4fd5-8c5c-cf9a0b26e160 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:57:07.820874 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e96e0-3ac2-4fd5-8c5c-cf9a0b26e160 service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:57:07.821155 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e96e0-3ac2-4fd5-8c5c-cf9a0b26e160 service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:57:07.821432 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e96e0-3ac2-4fd5-8c5c-cf9a0b26e160 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:57:07.823983 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-338e96e0-3ac2-4fd5-8c5c-cf9a0b26e160 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:57:07.836753 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e96e0-3ac2-4fd5-8c5c-cf9a0b26e160 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:57:07.838497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e96e0-3ac2-4fd5-8c5c-cf9a0b26e160 service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:57:07.851604 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1193/2353] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:57:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:57:12.020691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=36 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:57:17.847006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=32 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:57:21.821604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40a034-a037-4529-9362-dae13edff1e1 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:57:21.824412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-cf40a034-a037-4529-9362-dae13edff1e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:57:21.841484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40a034-a037-4529-9362-dae13edff1e1 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:57:21.845987 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeeb81c-291d-440a-86c4-960005a7e3a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:57:21.847037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeeb81c-291d-440a-86c4-960005a7e3a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:57:21.847591 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeeb81c-291d-440a-86c4-960005a7e3a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:57:21.848102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeeb81c-291d-440a-86c4-960005a7e3a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:57:21.851424 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-caeeb81c-291d-440a-86c4-960005a7e3a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:57:21.883714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-caeeb81c-291d-440a-86c4-960005a7e3a2 service cinder] Could not find project: 63125418cd9949cdbf76099952b31950.: keystone.exception.ProjectNotFound: Could not find project: 63125418cd9949cdbf76099952b31950. Aug 01 15:57:21.883714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:57:21.883714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:57:21.883714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:57:21.883714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:57:21.883714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:57:21.883714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:57:21.883714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:57:21.883714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:57:21.883714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:57:21.883714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:57:21.883714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:57:21.883714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Aug 01 15:57:21.883714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Aug 01 15:57:21.883714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Aug 01 15:57:21.886402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Aug 01 15:57:21.886402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Aug 01 15:57:21.886402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Aug 01 15:57:21.886402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:57:21.886402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:57:21.886402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Aug 01 15:57:21.886402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Aug 01 15:57:21.886402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:57:21.886402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:57:21.886402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:57:21.886402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:57:21.886402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:57:21.886402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:57:21.886402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:57:21.886402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:57:21.886402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:57:21.886402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:57:21.888411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:57:21.888411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:57:21.888411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:57:21.888411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:57:21.888411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Aug 01 15:57:21.888411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Aug 01 15:57:21.888411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Aug 01 15:57:21.888411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._validate_token_resources() Aug 01 15:57:21.888411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Aug 01 15:57:21.888411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Aug 01 15:57:21.888411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Aug 01 15:57:21.888411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Aug 01 15:57:21.888411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:57:21.888411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:57:21.888411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:57:21.888411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:57:21.888411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:57:21.891789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:57:21.891789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:57:21.891789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 15:57:21.891789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:57:21.891789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:57:21.891789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:57:21.891789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:57:21.891789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:57:21.891789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:57:21.891789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:57:21.891789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:57:21.891789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:57:21.891789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:57:21.891789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:57:21.891789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:57:21.891789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:57:21.891789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:57:21.897321 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 63125418cd9949cdbf76099952b31950. Aug 01 15:57:21.897321 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 15:57:21.897321 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1161/2354] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:57:21 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 66 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 01 15:57:22.081619 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e96e0-3ac2-4fd5-8c5c-cf9a0b26e160 service placement] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:57:22.083193 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-338e96e0-3ac2-4fd5-8c5c-cf9a0b26e160 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:57:22.197302 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e96e0-3ac2-4fd5-8c5c-cf9a0b26e160 service placement] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:57:22.210620 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2a05e-078d-460e-b2dc-51a1b8b4fe35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:57:22.212742 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2a05e-078d-460e-b2dc-51a1b8b4fe35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:57:22.213408 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2a05e-078d-460e-b2dc-51a1b8b4fe35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:57:22.214834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2a05e-078d-460e-b2dc-51a1b8b4fe35 admin admin] PATH_INFO: `/v3/users/3f56b135708f44218c36ecc39cdda65a` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:57:22.217730 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2a05e-078d-460e-b2dc-51a1b8b4fe35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f56b135708f44218c36ecc39cdda65a)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:57:22.222774 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2a05e-078d-460e-b2dc-51a1b8b4fe35 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:57:22.368940 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeeb81c-291d-440a-86c4-960005a7e3a2 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:57:22.402630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeeb81c-291d-440a-86c4-960005a7e3a2 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:57:22.413090 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1dc2a05e-078d-460e-b2dc-51a1b8b4fe35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f56b135708f44218c36ecc39cdda65a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:57:22.414824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1dc2a05e-078d-460e-b2dc-51a1b8b4fe35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f56b135708f44218c36ecc39cdda65a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:57:22.420444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d166b6d-dcc1-43d0-bd23-24c20e234239 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:57:22.421099 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d166b6d-dcc1-43d0-bd23-24c20e234239 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:57:22.421385 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d166b6d-dcc1-43d0-bd23-24c20e234239 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:57:22.421856 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d166b6d-dcc1-43d0-bd23-24c20e234239 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:57:22.428677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4d166b6d-dcc1-43d0-bd23-24c20e234239 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:57:22.447368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1dc2a05e-078d-460e-b2dc-51a1b8b4fe35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f56b135708f44218c36ecc39cdda65a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:57:22.481690 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1dc2a05e-078d-460e-b2dc-51a1b8b4fe35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f56b135708f44218c36ecc39cdda65a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:57:22.526148 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1dc2a05e-078d-460e-b2dc-51a1b8b4fe35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f56b135708f44218c36ecc39cdda65a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:57:22.540477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1194/2355] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 15:57:22 2022] DELETE /identity/v3/users/3f56b135708f44218c36ecc39cdda65a => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:57:22.546117 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d166b6d-dcc1-43d0-bd23-24c20e234239 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:57:22.547854 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddf375-b91b-43f0-9d30-b5cec94ad3ae None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:57:22.548256 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddf375-b91b-43f0-9d30-b5cec94ad3ae None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:57:22.548642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddf375-b91b-43f0-9d30-b5cec94ad3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:57:22.559800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d166b6d-dcc1-43d0-bd23-24c20e234239 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:57:22.593576 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-02ddf375-b91b-43f0-9d30-b5cec94ad3ae None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:57:22.594892 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-02ddf375-b91b-43f0-9d30-b5cec94ad3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:57:22.600761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1162/2356] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:57:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:57:22.708738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1195/2357] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:57:22.724623 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d166b6d-dcc1-43d0-bd23-24c20e234239 service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:57:22.727431 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4d166b6d-dcc1-43d0-bd23-24c20e234239 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:57:22.754541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d166b6d-dcc1-43d0-bd23-24c20e234239 service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:57:22.831167 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5081f7-8c32-4c60-9dd7-90d27a77bdf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:57:22.832995 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5081f7-8c32-4c60-9dd7-90d27a77bdf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:57:22.832995 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5081f7-8c32-4c60-9dd7-90d27a77bdf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:57:22.832995 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5081f7-8c32-4c60-9dd7-90d27a77bdf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:57:22.850914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5081f7-8c32-4c60-9dd7-90d27a77bdf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:57:22.852462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5081f7-8c32-4c60-9dd7-90d27a77bdf3 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:57:22.865746 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1163/2358] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:57:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:57:24.411178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ddf375-b91b-43f0-9d30-b5cec94ad3ae None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:57:24.413364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-02ddf375-b91b-43f0-9d30-b5cec94ad3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:57:24.433470 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ddf375-b91b-43f0-9d30-b5cec94ad3ae None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:57:24.446037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c007dbf-35c0-4e6d-a69a-6f31cda26c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:57:24.446973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c007dbf-35c0-4e6d-a69a-6f31cda26c34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:57:24.447438 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c007dbf-35c0-4e6d-a69a-6f31cda26c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:57:24.447903 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c007dbf-35c0-4e6d-a69a-6f31cda26c34 admin admin] PATH_INFO: `/v3/projects/c1c78d23cae448c7ad7771f924f858e4` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:57:24.450241 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c007dbf-35c0-4e6d-a69a-6f31cda26c34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1c78d23cae448c7ad7771f924f858e4)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:57:24.452932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c007dbf-35c0-4e6d-a69a-6f31cda26c34 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:57:24.465102 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:57:24.513547 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5c007dbf-35c0-4e6d-a69a-6f31cda26c34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1c78d23cae448c7ad7771f924f858e4', 'request_id': 'req-5c007dbf-35c0-4e6d-a69a-6f31cda26c34'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:57:24.526584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5c007dbf-35c0-4e6d-a69a-6f31cda26c34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1c78d23cae448c7ad7771f924f858e4', 'request_id': 'req-5c007dbf-35c0-4e6d-a69a-6f31cda26c34'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:57:24.534076 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5c007dbf-35c0-4e6d-a69a-6f31cda26c34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1c78d23cae448c7ad7771f924f858e4', 'request_id': 'req-5c007dbf-35c0-4e6d-a69a-6f31cda26c34'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:57:24.548851 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5c007dbf-35c0-4e6d-a69a-6f31cda26c34 admin admin] The token cache is being invalidate because project c1c78d23cae448c7ad7771f924f858e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:57:24.549138 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5c007dbf-35c0-4e6d-a69a-6f31cda26c34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:57:24.549785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5c007dbf-35c0-4e6d-a69a-6f31cda26c34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:57:24.551361 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1196/2359] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 15:57:24 2022] DELETE /identity/v3/projects/c1c78d23cae448c7ad7771f924f858e4 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:57:25.297107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5081f7-8c32-4c60-9dd7-90d27a77bdf3 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:57:25.303223 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7d5081f7-8c32-4c60-9dd7-90d27a77bdf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:57:25.486436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5081f7-8c32-4c60-9dd7-90d27a77bdf3 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:57:25.576871 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf0b42-b0e6-494a-96fb-474ff015b9f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:57:25.577574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf0b42-b0e6-494a-96fb-474ff015b9f7 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:57:25.577923 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf0b42-b0e6-494a-96fb-474ff015b9f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:57:25.578212 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf0b42-b0e6-494a-96fb-474ff015b9f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:57:25.580581 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6faf0b42-b0e6-494a-96fb-474ff015b9f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:57:25.590829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:57:25.598497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf0b42-b0e6-494a-96fb-474ff015b9f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:57:25.599922 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf0b42-b0e6-494a-96fb-474ff015b9f7 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:57:25.613541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1164/2360] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:57:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:57:25.897968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ce452-508e-4c3f-b1e8-ddfffd19a6af None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:57:25.899069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ce452-508e-4c3f-b1e8-ddfffd19a6af None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:57:25.899754 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ce452-508e-4c3f-b1e8-ddfffd19a6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:57:25.949255 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d48ce452-508e-4c3f-b1e8-ddfffd19a6af None None] MFA Rules not processed for user `4001e8ec57b34f57aae1654fad9217f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:57:25.951003 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d48ce452-508e-4c3f-b1e8-ddfffd19a6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:57:26.036602 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1197/2361] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:57:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:57:26.589245 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf0b42-b0e6-494a-96fb-474ff015b9f7 service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:57:26.590821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6faf0b42-b0e6-494a-96fb-474ff015b9f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:57:26.661827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf0b42-b0e6-494a-96fb-474ff015b9f7 service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:57:26.666383 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef2c89-cae7-4e2e-8108-5095432b87e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:57:26.667646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef2c89-cae7-4e2e-8108-5095432b87e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:57:26.667984 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef2c89-cae7-4e2e-8108-5095432b87e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:57:26.668535 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef2c89-cae7-4e2e-8108-5095432b87e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:57:26.670500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-88ef2c89-cae7-4e2e-8108-5095432b87e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:57:26.680693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef2c89-cae7-4e2e-8108-5095432b87e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:57:26.682193 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef2c89-cae7-4e2e-8108-5095432b87e1 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:57:26.691233 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1165/2362] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:57:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:57:27.163007 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ce452-508e-4c3f-b1e8-ddfffd19a6af None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:57:27.165376 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d48ce452-508e-4c3f-b1e8-ddfffd19a6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:57:27.267133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ce452-508e-4c3f-b1e8-ddfffd19a6af None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:57:27.273843 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c4620-91b1-4469-942f-cd582f5f7711 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:57:27.274968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c4620-91b1-4469-942f-cd582f5f7711 service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:57:27.275473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c4620-91b1-4469-942f-cd582f5f7711 service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:57:27.276002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c4620-91b1-4469-942f-cd582f5f7711 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:57:27.279756 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-874c4620-91b1-4469-942f-cd582f5f7711 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:57:27.294799 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c4620-91b1-4469-942f-cd582f5f7711 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:57:27.297257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c4620-91b1-4469-942f-cd582f5f7711 service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:57:27.314855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1198/2363] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:57:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:57:35.638772 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef2c89-cae7-4e2e-8108-5095432b87e1 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:57:35.640383 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-88ef2c89-cae7-4e2e-8108-5095432b87e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:57:35.658362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef2c89-cae7-4e2e-8108-5095432b87e1 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:57:35.665104 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b6d23-4a70-4ad0-9d4d-96653aacda09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:57:35.666156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b6d23-4a70-4ad0-9d4d-96653aacda09 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:57:35.666494 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b6d23-4a70-4ad0-9d4d-96653aacda09 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:57:35.667465 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b6d23-4a70-4ad0-9d4d-96653aacda09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:57:35.670584 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9b0b6d23-4a70-4ad0-9d4d-96653aacda09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:57:35.686589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b6d23-4a70-4ad0-9d4d-96653aacda09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:57:35.688139 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b6d23-4a70-4ad0-9d4d-96653aacda09 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:57:35.701382 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1166/2364] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:57:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:57:37.306611 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:57:45.695010 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=35 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:58:58.307529 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c4620-91b1-4469-942f-cd582f5f7711 service placement] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:58:58.317099 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c4620-91b1-4469-942f-cd582f5f7711 service placement] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:58:58.321265 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f00ad2-9c5c-473b-bf22-55cad1db927b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:58:58.322003 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f00ad2-9c5c-473b-bf22-55cad1db927b service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:58:58.322404 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f00ad2-9c5c-473b-bf22-55cad1db927b service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:58:58.322753 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f00ad2-9c5c-473b-bf22-55cad1db927b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:58:58.324648 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-17f00ad2-9c5c-473b-bf22-55cad1db927b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:58:58.339496 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f00ad2-9c5c-473b-bf22-55cad1db927b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:58:58.340970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f00ad2-9c5c-473b-bf22-55cad1db927b service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:58:58.355037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1199/2365] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:58:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:58:59.482050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b6d23-4a70-4ad0-9d4d-96653aacda09 service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:58:59.492822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b6d23-4a70-4ad0-9d4d-96653aacda09 service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:58:59.496911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2704fd0-1362-4c5d-be1c-6db236367462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:58:59.497590 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2704fd0-1362-4c5d-be1c-6db236367462 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:58:59.497875 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2704fd0-1362-4c5d-be1c-6db236367462 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:58:59.498160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2704fd0-1362-4c5d-be1c-6db236367462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:58:59.500000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d2704fd0-1362-4c5d-be1c-6db236367462 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:58:59.514733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2704fd0-1362-4c5d-be1c-6db236367462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:58:59.516216 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2704fd0-1362-4c5d-be1c-6db236367462 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:58:59.528408 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1167/2366] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:58:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:59:04.424125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679d0f4-47f5-4eaf-974c-135b90c862f9 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:59:04.424663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679d0f4-47f5-4eaf-974c-135b90c862f9 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:59:04.424663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679d0f4-47f5-4eaf-974c-135b90c862f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:59:04.454933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e679d0f4-47f5-4eaf-974c-135b90c862f9 None None] MFA Rules not processed for user `7d1b6c388c7642a892c905d16f3b8d50`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:59:04.456174 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e679d0f4-47f5-4eaf-974c-135b90c862f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:59:04.518036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1200/2367] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 15:59:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:59:04.833185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2704fd0-1362-4c5d-be1c-6db236367462 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:59:04.842521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2704fd0-1362-4c5d-be1c-6db236367462 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:59:04.846965 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886c44f-c607-4ffb-90d8-a8085cc7ccb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:59:04.847585 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886c44f-c607-4ffb-90d8-a8085cc7ccb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:59:04.847871 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886c44f-c607-4ffb-90d8-a8085cc7ccb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:59:04.848143 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886c44f-c607-4ffb-90d8-a8085cc7ccb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:59:04.849962 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e886c44f-c607-4ffb-90d8-a8085cc7ccb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:59:04.860688 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886c44f-c607-4ffb-90d8-a8085cc7ccb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:59:04.862218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886c44f-c607-4ffb-90d8-a8085cc7ccb5 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:59:04.870151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1168/2368] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:59:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:59:14.507318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=34 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:59:14.866650 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=12 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:59:19.993482 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679d0f4-47f5-4eaf-974c-135b90c862f9 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:59:19.994999 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e679d0f4-47f5-4eaf-974c-135b90c862f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:59:20.011447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679d0f4-47f5-4eaf-974c-135b90c862f9 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:59:20.019214 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf73be8-54d6-467e-b492-9a9dfa7c2f36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:59:20.020274 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf73be8-54d6-467e-b492-9a9dfa7c2f36 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:59:20.020781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf73be8-54d6-467e-b492-9a9dfa7c2f36 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:59:20.021065 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf73be8-54d6-467e-b492-9a9dfa7c2f36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:59:20.022985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6bf73be8-54d6-467e-b492-9a9dfa7c2f36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:59:20.049414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf73be8-54d6-467e-b492-9a9dfa7c2f36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:59:20.051062 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf73be8-54d6-467e-b492-9a9dfa7c2f36 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:59:20.061073 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1201/2369] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:59:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:59:25.487567 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886c44f-c607-4ffb-90d8-a8085cc7ccb5 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:59:25.488981 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e886c44f-c607-4ffb-90d8-a8085cc7ccb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:59:25.558352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886c44f-c607-4ffb-90d8-a8085cc7ccb5 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:59:25.561996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199b4ec-2cc8-408b-bee6-58bb80aa2db0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:59:25.562621 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199b4ec-2cc8-408b-bee6-58bb80aa2db0 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:59:25.562906 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199b4ec-2cc8-408b-bee6-58bb80aa2db0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:59:25.563179 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199b4ec-2cc8-408b-bee6-58bb80aa2db0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:59:25.564950 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1199b4ec-2cc8-408b-bee6-58bb80aa2db0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:59:25.588386 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199b4ec-2cc8-408b-bee6-58bb80aa2db0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:59:25.589874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199b4ec-2cc8-408b-bee6-58bb80aa2db0 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:59:25.600384 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1169/2370] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 15:59:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:59:30.056390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=11 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:59:35.595845 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=36 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:00:01.477104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf73be8-54d6-467e-b492-9a9dfa7c2f36 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:00:01.489503 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf73be8-54d6-467e-b492-9a9dfa7c2f36 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:00:01.495466 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792ee12-c457-48e3-ad04-e16854188932 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:00:01.496080 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792ee12-c457-48e3-ad04-e16854188932 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:00:01.496372 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792ee12-c457-48e3-ad04-e16854188932 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:00:01.496644 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792ee12-c457-48e3-ad04-e16854188932 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:00:01.498482 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e792ee12-c457-48e3-ad04-e16854188932 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:00:01.570728 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792ee12-c457-48e3-ad04-e16854188932 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:00:01.572314 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792ee12-c457-48e3-ad04-e16854188932 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:00:01.581790 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1202/2371] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:00:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:00:11.577069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=33 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:01:02.721822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199b4ec-2cc8-408b-bee6-58bb80aa2db0 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:01:02.723288 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1199b4ec-2cc8-408b-bee6-58bb80aa2db0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:01:02.735112 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199b4ec-2cc8-408b-bee6-58bb80aa2db0 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:01:02.738764 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49c2de-d7d3-470d-95b9-6dabc577fe9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:01:02.739378 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49c2de-d7d3-470d-95b9-6dabc577fe9f service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:01:02.739664 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49c2de-d7d3-470d-95b9-6dabc577fe9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:01:02.739954 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49c2de-d7d3-470d-95b9-6dabc577fe9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:01:02.741650 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-cb49c2de-d7d3-470d-95b9-6dabc577fe9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:01:02.813371 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49c2de-d7d3-470d-95b9-6dabc577fe9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:01:02.814774 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49c2de-d7d3-470d-95b9-6dabc577fe9f service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:01:02.824517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1170/2372] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:01:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:01:07.356570 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792ee12-c457-48e3-ad04-e16854188932 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:01:07.358058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e792ee12-c457-48e3-ad04-e16854188932 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:01:07.381301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792ee12-c457-48e3-ad04-e16854188932 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:01:07.385425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a8abb-7542-4d2c-9860-c1a1a297c996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:01:07.386077 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a8abb-7542-4d2c-9860-c1a1a297c996 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:01:07.386360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a8abb-7542-4d2c-9860-c1a1a297c996 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:01:07.386634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a8abb-7542-4d2c-9860-c1a1a297c996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:01:07.388985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4a1a8abb-7542-4d2c-9860-c1a1a297c996 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:01:07.398365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a8abb-7542-4d2c-9860-c1a1a297c996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:01:07.400359 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a8abb-7542-4d2c-9860-c1a1a297c996 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:01:07.408289 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1203/2373] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:01:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:01:12.820485 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=33 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:01:17.404742 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=11 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:02:02.204309 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49c2de-d7d3-470d-95b9-6dabc577fe9f service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:02:02.238943 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49c2de-d7d3-470d-95b9-6dabc577fe9f service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:02:02.245414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70905d90-dc46-40c1-aa5c-c317ddf354d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:02:02.246192 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70905d90-dc46-40c1-aa5c-c317ddf354d0 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:02:02.246372 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70905d90-dc46-40c1-aa5c-c317ddf354d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:02:02.246636 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70905d90-dc46-40c1-aa5c-c317ddf354d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:02:02.248444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-70905d90-dc46-40c1-aa5c-c317ddf354d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:02:02.330361 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70905d90-dc46-40c1-aa5c-c317ddf354d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:02:02.331892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70905d90-dc46-40c1-aa5c-c317ddf354d0 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:02:02.341543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1171/2374] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:02:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:02:11.644499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a8abb-7542-4d2c-9860-c1a1a297c996 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:02:11.645904 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4a1a8abb-7542-4d2c-9860-c1a1a297c996 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:02:11.690832 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a8abb-7542-4d2c-9860-c1a1a297c996 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:02:11.695869 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcd183-84ad-4949-a1a4-bd013a3260d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:02:11.696481 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcd183-84ad-4949-a1a4-bd013a3260d4 service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:02:11.696760 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcd183-84ad-4949-a1a4-bd013a3260d4 service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:02:11.697034 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcd183-84ad-4949-a1a4-bd013a3260d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:02:11.698769 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-49dcd183-84ad-4949-a1a4-bd013a3260d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:02:11.710544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcd183-84ad-4949-a1a4-bd013a3260d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:02:11.712114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcd183-84ad-4949-a1a4-bd013a3260d4 service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:02:11.721914 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1204/2375] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:02:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:02:12.337410 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=46 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:02:21.717915 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=19 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:03:25.230476 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37a309-a518-438c-be47-47e6f84c5ba2 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:25.231042 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37a309-a518-438c-be47-47e6f84c5ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:25.231414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37a309-a518-438c-be47-47e6f84c5ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:25.259518 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-1e37a309-a518-438c-be47-47e6f84c5ba2 None None] MFA Rules not processed for user `ba149d1623e849328e399d6dbdca3e0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:03:25.260745 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1e37a309-a518-438c-be47-47e6f84c5ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:03:25.268529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1172/2376] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:03:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:03:26.891584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcd183-84ad-4949-a1a4-bd013a3260d4 service placement] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:03:26.915862 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcd183-84ad-4949-a1a4-bd013a3260d4 service placement] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:03:26.922484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b28dd2-4799-416c-acb6-b8b72bea87fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:03:26.923114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b28dd2-4799-416c-acb6-b8b72bea87fc service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:26.923491 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b28dd2-4799-416c-acb6-b8b72bea87fc service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:26.923653 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b28dd2-4799-416c-acb6-b8b72bea87fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:26.935104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b28dd2-4799-416c-acb6-b8b72bea87fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:03:26.937010 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b28dd2-4799-416c-acb6-b8b72bea87fc service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:03:26.949832 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1205/2377] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:03:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:03:27.331724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37a309-a518-438c-be47-47e6f84c5ba2 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:03:27.352951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37a309-a518-438c-be47-47e6f84c5ba2 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:03:27.359553 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db8eb9-4443-4b21-b580-cf694a670494 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:03:27.360485 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db8eb9-4443-4b21-b580-cf694a670494 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:27.360949 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db8eb9-4443-4b21-b580-cf694a670494 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:27.361403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db8eb9-4443-4b21-b580-cf694a670494 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:27.372689 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db8eb9-4443-4b21-b580-cf694a670494 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:03:27.374737 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db8eb9-4443-4b21-b580-cf694a670494 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:03:27.398283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1173/2378] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:03:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:03:29.771079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b28dd2-4799-416c-acb6-b8b72bea87fc service placement] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:03:29.797092 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b28dd2-4799-416c-acb6-b8b72bea87fc service placement] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:03:29.803714 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da303d93-fc9b-45a0-af94-897f64dcde84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:03:29.804487 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da303d93-fc9b-45a0-af94-897f64dcde84 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:29.804875 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da303d93-fc9b-45a0-af94-897f64dcde84 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:29.805381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da303d93-fc9b-45a0-af94-897f64dcde84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:29.812755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-da303d93-fc9b-45a0-af94-897f64dcde84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:03:29.860750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da303d93-fc9b-45a0-af94-897f64dcde84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:03:29.862478 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da303d93-fc9b-45a0-af94-897f64dcde84 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:03:29.876269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1206/2379] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:03:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:03:31.069361 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db8eb9-4443-4b21-b580-cf694a670494 service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:03:31.079317 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db8eb9-4443-4b21-b580-cf694a670494 service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:03:31.083093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c2036-7004-4f7c-b927-5d74c90df49b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:03:31.083708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c2036-7004-4f7c-b927-5d74c90df49b service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:31.083994 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c2036-7004-4f7c-b927-5d74c90df49b service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:31.084273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c2036-7004-4f7c-b927-5d74c90df49b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:31.086089 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f39c2036-7004-4f7c-b927-5d74c90df49b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:03:31.171219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c2036-7004-4f7c-b927-5d74c90df49b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:03:31.172667 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c2036-7004-4f7c-b927-5d74c90df49b service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:03:31.191331 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1174/2380] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:03:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:03:34.691547 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da303d93-fc9b-45a0-af94-897f64dcde84 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:03:34.705453 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da303d93-fc9b-45a0-af94-897f64dcde84 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:03:34.711625 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55579b87-ce83-41f5-b464-0889922efd88 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:03:34.712476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55579b87-ce83-41f5-b464-0889922efd88 service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:34.712945 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55579b87-ce83-41f5-b464-0889922efd88 service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:34.713391 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55579b87-ce83-41f5-b464-0889922efd88 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:34.716079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-55579b87-ce83-41f5-b464-0889922efd88 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:03:34.731861 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55579b87-ce83-41f5-b464-0889922efd88 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:03:34.733482 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55579b87-ce83-41f5-b464-0889922efd88 service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:03:34.753860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1207/2381] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:03:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:03:34.769378 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c2036-7004-4f7c-b927-5d74c90df49b service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:03:34.771143 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f39c2036-7004-4f7c-b927-5d74c90df49b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:03:34.795240 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c2036-7004-4f7c-b927-5d74c90df49b service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:03:34.804186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106d5c9-93af-44d8-b777-be738a193a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:03:34.805059 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106d5c9-93af-44d8-b777-be738a193a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:34.806201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106d5c9-93af-44d8-b777-be738a193a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:34.806499 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106d5c9-93af-44d8-b777-be738a193a36 admin admin] PATH_INFO: `/v3/users/5f0b12ba782244d7b44783c3f3532bfa` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:34.807998 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c106d5c9-93af-44d8-b777-be738a193a36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f0b12ba782244d7b44783c3f3532bfa)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:03:34.810260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c106d5c9-93af-44d8-b777-be738a193a36 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:03:34.864364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c106d5c9-93af-44d8-b777-be738a193a36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f0b12ba782244d7b44783c3f3532bfa'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:03:34.865018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c106d5c9-93af-44d8-b777-be738a193a36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f0b12ba782244d7b44783c3f3532bfa'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:03:34.872903 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c106d5c9-93af-44d8-b777-be738a193a36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f0b12ba782244d7b44783c3f3532bfa'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:03:34.886235 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c106d5c9-93af-44d8-b777-be738a193a36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f0b12ba782244d7b44783c3f3532bfa'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:03:34.887483 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c106d5c9-93af-44d8-b777-be738a193a36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f0b12ba782244d7b44783c3f3532bfa'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:03:34.900169 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1175/2382] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:03:34 2022] DELETE /identity/v3/users/5f0b12ba782244d7b44783c3f3532bfa => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:03:35.099957 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55579b87-ce83-41f5-b464-0889922efd88 service placement] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:03:35.113431 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55579b87-ce83-41f5-b464-0889922efd88 service placement] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:03:35.187471 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aac563-9b9e-4e9a-825a-98394ef6f96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:03:35.188236 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aac563-9b9e-4e9a-825a-98394ef6f96a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:35.188586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aac563-9b9e-4e9a-825a-98394ef6f96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:35.188931 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aac563-9b9e-4e9a-825a-98394ef6f96a admin admin] PATH_INFO: `/v3/projects/2d49f0b3378c42ada96c61586a38136e` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:35.191168 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aac563-9b9e-4e9a-825a-98394ef6f96a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d49f0b3378c42ada96c61586a38136e)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:03:35.193258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aac563-9b9e-4e9a-825a-98394ef6f96a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:03:35.245973 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106d5c9-93af-44d8-b777-be738a193a36 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:03:35.246206 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b7aac563-9b9e-4e9a-825a-98394ef6f96a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d49f0b3378c42ada96c61586a38136e', 'request_id': 'req-b7aac563-9b9e-4e9a-825a-98394ef6f96a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:03:35.248474 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c106d5c9-93af-44d8-b777-be738a193a36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:03:35.253770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b7aac563-9b9e-4e9a-825a-98394ef6f96a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d49f0b3378c42ada96c61586a38136e', 'request_id': 'req-b7aac563-9b9e-4e9a-825a-98394ef6f96a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:03:35.259691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:03:35.261210 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b7aac563-9b9e-4e9a-825a-98394ef6f96a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d49f0b3378c42ada96c61586a38136e', 'request_id': 'req-b7aac563-9b9e-4e9a-825a-98394ef6f96a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:03:35.276296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b7aac563-9b9e-4e9a-825a-98394ef6f96a admin admin] The token cache is being invalidate because project 2d49f0b3378c42ada96c61586a38136e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:03:35.276582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b7aac563-9b9e-4e9a-825a-98394ef6f96a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:03:35.277240 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b7aac563-9b9e-4e9a-825a-98394ef6f96a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:03:35.279107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1208/2383] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:03:35 2022] DELETE /identity/v3/projects/2d49f0b3378c42ada96c61586a38136e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:03:35.284519 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea297a-7577-4de6-97b2-607c6b31558e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:35.284878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea297a-7577-4de6-97b2-607c6b31558e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:35.285231 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea297a-7577-4de6-97b2-607c6b31558e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:35.337520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:03:35.340342 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106d5c9-93af-44d8-b777-be738a193a36 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:03:35.344442 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c4ea297a-7577-4de6-97b2-607c6b31558e None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:03:35.346917 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c4ea297a-7577-4de6-97b2-607c6b31558e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:03:35.406572 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84186239-cd98-4ddf-b9a3-4183884f3391 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:03:35.407229 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84186239-cd98-4ddf-b9a3-4183884f3391 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:35.407525 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84186239-cd98-4ddf-b9a3-4183884f3391 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:35.407805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84186239-cd98-4ddf-b9a3-4183884f3391 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:35.409296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1209/2384] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:03:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:03:35.409708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-84186239-cd98-4ddf-b9a3-4183884f3391 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:03:35.417677 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b2a5a-3e44-4863-a5e6-d5d652034fea None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:35.418120 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b2a5a-3e44-4863-a5e6-d5d652034fea None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:35.418969 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b2a5a-3e44-4863-a5e6-d5d652034fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:35.423263 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84186239-cd98-4ddf-b9a3-4183884f3391 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:03:35.424997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84186239-cd98-4ddf-b9a3-4183884f3391 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:03:35.436446 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1176/2385] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:03:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:03:35.440013 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-966b2a5a-3e44-4863-a5e6-d5d652034fea None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:03:35.441216 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-966b2a5a-3e44-4863-a5e6-d5d652034fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:03:35.449319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1210/2386] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:03:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:03:35.453613 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84186239-cd98-4ddf-b9a3-4183884f3391 service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:03:35.464611 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84186239-cd98-4ddf-b9a3-4183884f3391 service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:03:35.470773 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e76ec7-c550-402c-807e-a146660987af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:03:35.471392 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e76ec7-c550-402c-807e-a146660987af admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:35.471704 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e76ec7-c550-402c-807e-a146660987af admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:35.471976 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e76ec7-c550-402c-807e-a146660987af admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:35.472674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e76ec7-c550-402c-807e-a146660987af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:03:35.473002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e76ec7-c550-402c-807e-a146660987af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:03:35.474632 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e76ec7-c550-402c-807e-a146660987af admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:03:35.489277 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1177/2387] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:03:35.494045 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b2a5a-3e44-4863-a5e6-d5d652034fea None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:03:35.504552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b2a5a-3e44-4863-a5e6-d5d652034fea None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:03:35.509304 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181f885-f578-40de-b4c9-004c3983f0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:03:35.509953 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181f885-f578-40de-b4c9-004c3983f0e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:35.510236 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181f885-f578-40de-b4c9-004c3983f0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:35.510508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181f885-f578-40de-b4c9-004c3983f0e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:35.511232 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181f885-f578-40de-b4c9-004c3983f0e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:03:35.513171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181f885-f578-40de-b4c9-004c3983f0e8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:03:35.552224 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1211/2388] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:03:35 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:03:35.556538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e76ec7-c550-402c-807e-a146660987af admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:03:35.568332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e76ec7-c550-402c-807e-a146660987af admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:03:35.627209 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f189c17-3e69-4bf7-a939-cd2a822284cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:03:35.627847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f189c17-3e69-4bf7-a939-cd2a822284cf admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:35.628145 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f189c17-3e69-4bf7-a939-cd2a822284cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:35.628413 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f189c17-3e69-4bf7-a939-cd2a822284cf admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:35.629099 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f189c17-3e69-4bf7-a939-cd2a822284cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:03:35.631097 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f189c17-3e69-4bf7-a939-cd2a822284cf admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:03:35.657750 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1178/2389] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:03:35 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:03:35.659119 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181f885-f578-40de-b4c9-004c3983f0e8 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:03:35.670193 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181f885-f578-40de-b4c9-004c3983f0e8 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:03:35.680375 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ab543-d20e-4ccd-9993-cb8c68cd7c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:03:35.681041 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ab543-d20e-4ccd-9993-cb8c68cd7c96 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:35.681302 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ab543-d20e-4ccd-9993-cb8c68cd7c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:35.681581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ab543-d20e-4ccd-9993-cb8c68cd7c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:35.682946 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ab543-d20e-4ccd-9993-cb8c68cd7c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:03:35.685117 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ab543-d20e-4ccd-9993-cb8c68cd7c96 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:03:35.695877 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1212/2390] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:03:35.699840 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f189c17-3e69-4bf7-a939-cd2a822284cf admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:03:35.710228 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f189c17-3e69-4bf7-a939-cd2a822284cf admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:03:35.715719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f167cf-b0cd-4b2f-b629-54094277452e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:03:35.716352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f167cf-b0cd-4b2f-b629-54094277452e admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:35.716634 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f167cf-b0cd-4b2f-b629-54094277452e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:35.716905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f167cf-b0cd-4b2f-b629-54094277452e admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:35.717657 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f167cf-b0cd-4b2f-b629-54094277452e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:03:35.719326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f167cf-b0cd-4b2f-b629-54094277452e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:03:35.720769 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:03:35.733357 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1179/2391] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:03:35.737309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ab543-d20e-4ccd-9993-cb8c68cd7c96 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:03:35.747832 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ab543-d20e-4ccd-9993-cb8c68cd7c96 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:03:35.752432 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ad68d-31f5-493c-b8a4-81443211ac0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:03:35.753083 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ad68d-31f5-493c-b8a4-81443211ac0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:35.753365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ad68d-31f5-493c-b8a4-81443211ac0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:35.753640 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ad68d-31f5-493c-b8a4-81443211ac0e admin admin] PATH_INFO: `/v3/projects/07e6cfce1a9a480fa039a0158683103f/users/1678f28de9e04a80bab033fd76fcd8b7/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:35.768979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ad68d-31f5-493c-b8a4-81443211ac0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=07e6cfce1a9a480fa039a0158683103f, user_id=1678f28de9e04a80bab033fd76fcd8b7, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:03:35.771680 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ad68d-31f5-493c-b8a4-81443211ac0e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:03:35.784996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1213/2392] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:03:35 2022] PUT /identity/v3/projects/07e6cfce1a9a480fa039a0158683103f/users/1678f28de9e04a80bab033fd76fcd8b7/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:03:35.796074 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdea92-3863-4ab7-8090-dd84afedc420 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:35.796591 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdea92-3863-4ab7-8090-dd84afedc420 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:35.796741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdea92-3863-4ab7-8090-dd84afedc420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:35.826212 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-59bdea92-3863-4ab7-8090-dd84afedc420 None None] MFA Rules not processed for user `1678f28de9e04a80bab033fd76fcd8b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:03:35.827402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-59bdea92-3863-4ab7-8090-dd84afedc420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:03:35.893150 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1180/2393] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:03:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:03:35.897843 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ad68d-31f5-493c-b8a4-81443211ac0e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:03:35.963210 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ad68d-31f5-493c-b8a4-81443211ac0e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:03:35.967884 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4d24f-cefe-4f06-b237-a2274bc54c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:03:35.968500 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4d24f-cefe-4f06-b237-a2274bc54c42 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:35.968776 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4d24f-cefe-4f06-b237-a2274bc54c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:35.969042 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4d24f-cefe-4f06-b237-a2274bc54c42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:35.969705 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4d24f-cefe-4f06-b237-a2274bc54c42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:03:35.971621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4d24f-cefe-4f06-b237-a2274bc54c42 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:03:35.999268 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1214/2394] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:03:35 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:03:36.003179 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdea92-3863-4ab7-8090-dd84afedc420 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:03:36.014800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdea92-3863-4ab7-8090-dd84afedc420 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:03:36.071007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436c306-63e0-4909-ac09-06f6b3912d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:03:36.071619 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436c306-63e0-4909-ac09-06f6b3912d1b admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:36.071907 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436c306-63e0-4909-ac09-06f6b3912d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:36.072209 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436c306-63e0-4909-ac09-06f6b3912d1b admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:36.072886 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436c306-63e0-4909-ac09-06f6b3912d1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:03:36.075493 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436c306-63e0-4909-ac09-06f6b3912d1b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:03:36.096569 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1181/2395] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:03:36 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:03:36.100697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4d24f-cefe-4f06-b237-a2274bc54c42 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:03:36.111914 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4d24f-cefe-4f06-b237-a2274bc54c42 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:03:36.116624 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85071f-7830-4cae-909c-36134fcbfcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:03:36.117338 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85071f-7830-4cae-909c-36134fcbfcb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:36.117625 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85071f-7830-4cae-909c-36134fcbfcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:36.117914 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85071f-7830-4cae-909c-36134fcbfcb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:36.118610 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85071f-7830-4cae-909c-36134fcbfcb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:03:36.120252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85071f-7830-4cae-909c-36134fcbfcb7 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:03:36.132586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1215/2396] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:03:36.136850 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436c306-63e0-4909-ac09-06f6b3912d1b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:03:36.147358 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436c306-63e0-4909-ac09-06f6b3912d1b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:03:36.152311 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593806f-7a4e-4cb6-9476-91c5b532cdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:03:36.153027 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593806f-7a4e-4cb6-9476-91c5b532cdaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:36.153312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593806f-7a4e-4cb6-9476-91c5b532cdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:36.153591 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593806f-7a4e-4cb6-9476-91c5b532cdaa admin admin] PATH_INFO: `/v3/projects/9636fc5cb75b421496fdd49fc2779edb/users/1ff6a9c6a2174a37a0f2bafba55dc987/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:36.169795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1593806f-7a4e-4cb6-9476-91c5b532cdaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=9636fc5cb75b421496fdd49fc2779edb, user_id=1ff6a9c6a2174a37a0f2bafba55dc987, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:03:36.172593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1593806f-7a4e-4cb6-9476-91c5b532cdaa admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:03:36.185998 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1182/2397] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:03:36 2022] PUT /identity/v3/projects/9636fc5cb75b421496fdd49fc2779edb/users/1ff6a9c6a2174a37a0f2bafba55dc987/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:03:36.190159 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85071f-7830-4cae-909c-36134fcbfcb7 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:03:36.200399 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85071f-7830-4cae-909c-36134fcbfcb7 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:03:36.205056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:03:36.260039 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bba682-5caf-458f-8ee6-2876bb0d80bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:03:36.260762 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bba682-5caf-458f-8ee6-2876bb0d80bc admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:36.261059 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bba682-5caf-458f-8ee6-2876bb0d80bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:36.261319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bba682-5caf-458f-8ee6-2876bb0d80bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:36.262067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bba682-5caf-458f-8ee6-2876bb0d80bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:03:36.263668 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bba682-5caf-458f-8ee6-2876bb0d80bc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:03:36.274793 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1216/2398] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:03:36.278998 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593806f-7a4e-4cb6-9476-91c5b532cdaa admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:03:36.289812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593806f-7a4e-4cb6-9476-91c5b532cdaa admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:03:36.295018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c92881-e97f-449d-8c70-e4c90599cfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:03:36.295688 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c92881-e97f-449d-8c70-e4c90599cfe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:36.295991 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c92881-e97f-449d-8c70-e4c90599cfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:36.296246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c92881-e97f-449d-8c70-e4c90599cfe6 admin admin] PATH_INFO: `/v3/projects/9636fc5cb75b421496fdd49fc2779edb/users/1ff6a9c6a2174a37a0f2bafba55dc987/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:36.298253 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c92881-e97f-449d-8c70-e4c90599cfe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9636fc5cb75b421496fdd49fc2779edb, user_id=1ff6a9c6a2174a37a0f2bafba55dc987, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:03:36.300991 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c92881-e97f-449d-8c70-e4c90599cfe6 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:03:36.313078 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1183/2399] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:03:36 2022] PUT /identity/v3/projects/9636fc5cb75b421496fdd49fc2779edb/users/1ff6a9c6a2174a37a0f2bafba55dc987/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:03:36.324670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a2b9c-d5e2-4fa5-a67d-1fc5d855ef3f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:03:36.324963 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a2b9c-d5e2-4fa5-a67d-1fc5d855ef3f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:03:36.325235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a2b9c-d5e2-4fa5-a67d-1fc5d855ef3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:03:36.355145 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-048a2b9c-d5e2-4fa5-a67d-1fc5d855ef3f None None] MFA Rules not processed for user `1ff6a9c6a2174a37a0f2bafba55dc987`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:03:36.357178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-048a2b9c-d5e2-4fa5-a67d-1fc5d855ef3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:03:36.433179 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1217/2400] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:03:36 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:03:46.308399 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=78,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:03:46.430301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=59 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:05:03.672550 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c92881-e97f-449d-8c70-e4c90599cfe6 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:05:03.675105 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-26c92881-e97f-449d-8c70-e4c90599cfe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:05:03.752009 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c92881-e97f-449d-8c70-e4c90599cfe6 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:05:03.756947 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c36ad-77b5-4a40-9322-f226947d66ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:05:03.758308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c36ad-77b5-4a40-9322-f226947d66ec service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:05:03.758730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c36ad-77b5-4a40-9322-f226947d66ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:05:03.759221 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c36ad-77b5-4a40-9322-f226947d66ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:05:03.762204 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-841c36ad-77b5-4a40-9322-f226947d66ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:05:03.845024 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c36ad-77b5-4a40-9322-f226947d66ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:05:03.847089 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c36ad-77b5-4a40-9322-f226947d66ec service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:05:03.866527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1184/2401] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:05:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:05:13.863745 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=60 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:05:22.821457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a2b9c-d5e2-4fa5-a67d-1fc5d855ef3f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:05:22.823465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-048a2b9c-d5e2-4fa5-a67d-1fc5d855ef3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:05:22.954533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a2b9c-d5e2-4fa5-a67d-1fc5d855ef3f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:05:22.963002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49f788-71a1-4368-8b21-21e1b4bd4ec2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:05:22.964006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49f788-71a1-4368-8b21-21e1b4bd4ec2 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:05:22.964458 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49f788-71a1-4368-8b21-21e1b4bd4ec2 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:05:22.964964 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49f788-71a1-4368-8b21-21e1b4bd4ec2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:05:22.967953 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5d49f788-71a1-4368-8b21-21e1b4bd4ec2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:05:23.047963 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49f788-71a1-4368-8b21-21e1b4bd4ec2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:05:23.049589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49f788-71a1-4368-8b21-21e1b4bd4ec2 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:05:23.073133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1218/2402] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:05:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:05:23.266572 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c36ad-77b5-4a40-9322-f226947d66ec service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:05:23.269400 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-841c36ad-77b5-4a40-9322-f226947d66ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:05:23.344290 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c36ad-77b5-4a40-9322-f226947d66ec service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:05:23.352299 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89461810-42b8-41e4-89ca-f558a19568c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:05:23.354041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89461810-42b8-41e4-89ca-f558a19568c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:05:23.354378 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89461810-42b8-41e4-89ca-f558a19568c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:05:23.354750 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89461810-42b8-41e4-89ca-f558a19568c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:05:23.356680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-89461810-42b8-41e4-89ca-f558a19568c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:05:23.373232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89461810-42b8-41e4-89ca-f558a19568c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:05:23.374794 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89461810-42b8-41e4-89ca-f558a19568c0 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:05:23.387469 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1185/2403] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:05:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:05:24.577225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49f788-71a1-4368-8b21-21e1b4bd4ec2 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:05:24.588484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49f788-71a1-4368-8b21-21e1b4bd4ec2 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:05:24.592144 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ff3b3-4f3c-471e-902d-7699f9e67390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:05:24.592773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ff3b3-4f3c-471e-902d-7699f9e67390 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:05:24.593057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ff3b3-4f3c-471e-902d-7699f9e67390 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:05:24.593332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ff3b3-4f3c-471e-902d-7699f9e67390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:05:24.595192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-715ff3b3-4f3c-471e-902d-7699f9e67390 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:05:24.610480 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ff3b3-4f3c-471e-902d-7699f9e67390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:05:24.611911 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ff3b3-4f3c-471e-902d-7699f9e67390 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:05:24.622982 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1219/2404] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:05:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:05:26.230914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89461810-42b8-41e4-89ca-f558a19568c0 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:05:26.232898 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-89461810-42b8-41e4-89ca-f558a19568c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:05:26.248999 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89461810-42b8-41e4-89ca-f558a19568c0 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:05:26.254429 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91e7f0-66ee-4683-a0da-79b10d4ab51e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:05:26.255073 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91e7f0-66ee-4683-a0da-79b10d4ab51e service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:05:26.255361 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91e7f0-66ee-4683-a0da-79b10d4ab51e service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:05:26.255637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91e7f0-66ee-4683-a0da-79b10d4ab51e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:05:26.257588 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-dc91e7f0-66ee-4683-a0da-79b10d4ab51e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:05:26.347980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91e7f0-66ee-4683-a0da-79b10d4ab51e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:05:26.349445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91e7f0-66ee-4683-a0da-79b10d4ab51e service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:05:26.372797 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1186/2405] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:05:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:05:26.435628 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ff3b3-4f3c-471e-902d-7699f9e67390 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:05:26.446593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ff3b3-4f3c-471e-902d-7699f9e67390 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:05:26.450660 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8921ad-7f6d-44dc-8ed0-705484b3e427 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:05:26.451423 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8921ad-7f6d-44dc-8ed0-705484b3e427 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:05:26.451731 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8921ad-7f6d-44dc-8ed0-705484b3e427 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:05:26.452012 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8921ad-7f6d-44dc-8ed0-705484b3e427 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:05:26.463389 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8921ad-7f6d-44dc-8ed0-705484b3e427 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:05:26.464849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8921ad-7f6d-44dc-8ed0-705484b3e427 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:05:26.478240 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1220/2406] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:05:26 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:05:31.804405 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ccce3-17e0-4732-91b8-7642eb295fe7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:05:31.804968 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ccce3-17e0-4732-91b8-7642eb295fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:05:31.805609 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ccce3-17e0-4732-91b8-7642eb295fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:05:31.829592 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-de8ccce3-17e0-4732-91b8-7642eb295fe7 None None] MFA Rules not processed for user `4001e8ec57b34f57aae1654fad9217f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:05:31.830890 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-de8ccce3-17e0-4732-91b8-7642eb295fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:05:31.838436 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1187/2407] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:05:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:05:32.173102 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8921ad-7f6d-44dc-8ed0-705484b3e427 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:05:32.184195 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8921ad-7f6d-44dc-8ed0-705484b3e427 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:05:32.188097 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d77f6f-45e7-492b-939a-19a7ebd1bacd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:05:32.188714 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d77f6f-45e7-492b-939a-19a7ebd1bacd service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:05:32.188990 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d77f6f-45e7-492b-939a-19a7ebd1bacd service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:05:32.189263 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d77f6f-45e7-492b-939a-19a7ebd1bacd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:05:32.191083 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-65d77f6f-45e7-492b-939a-19a7ebd1bacd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:05:32.201298 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d77f6f-45e7-492b-939a-19a7ebd1bacd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:05:32.202762 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d77f6f-45e7-492b-939a-19a7ebd1bacd service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:05:32.211603 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1221/2408] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:05:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:05:41.823708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=69 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:05:42.207559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:06:13.290591 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ccce3-17e0-4732-91b8-7642eb295fe7 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:06:13.292319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-de8ccce3-17e0-4732-91b8-7642eb295fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:06:13.305466 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ccce3-17e0-4732-91b8-7642eb295fe7 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:06:13.309245 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb38d8-7edc-4ae0-b955-e18e09fc4e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:06:13.309878 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb38d8-7edc-4ae0-b955-e18e09fc4e49 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:06:13.310206 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb38d8-7edc-4ae0-b955-e18e09fc4e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:06:13.310486 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb38d8-7edc-4ae0-b955-e18e09fc4e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:06:13.312330 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f6bb38d8-7edc-4ae0-b955-e18e09fc4e49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:06:13.322958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb38d8-7edc-4ae0-b955-e18e09fc4e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:06:13.324457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb38d8-7edc-4ae0-b955-e18e09fc4e49 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:06:13.334062 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1188/2409] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:06:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:06:23.329633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=6 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:07:03.903403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d77f6f-45e7-492b-939a-19a7ebd1bacd service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:07:03.913471 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d77f6f-45e7-492b-939a-19a7ebd1bacd service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:07:03.917245 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a650e-b8d7-4674-9b8a-2fe7086b9e75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:07:03.917861 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a650e-b8d7-4674-9b8a-2fe7086b9e75 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:07:03.918178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a650e-b8d7-4674-9b8a-2fe7086b9e75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:07:03.918483 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a650e-b8d7-4674-9b8a-2fe7086b9e75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:07:03.920251 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b47a650e-b8d7-4674-9b8a-2fe7086b9e75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:07:03.992583 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a650e-b8d7-4674-9b8a-2fe7086b9e75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:07:03.994056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a650e-b8d7-4674-9b8a-2fe7086b9e75 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:07:04.012170 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1222/2410] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:07:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:07:14.009368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=36 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:07:14.291902 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb38d8-7edc-4ae0-b955-e18e09fc4e49 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:07:14.293352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f6bb38d8-7edc-4ae0-b955-e18e09fc4e49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:07:14.365700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb38d8-7edc-4ae0-b955-e18e09fc4e49 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:07:14.370904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11044ae9-5b55-44ff-8d70-25fba1b363f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:07:14.371535 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11044ae9-5b55-44ff-8d70-25fba1b363f9 service placement] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:07:14.371820 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11044ae9-5b55-44ff-8d70-25fba1b363f9 service placement] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:07:14.372095 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11044ae9-5b55-44ff-8d70-25fba1b363f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:07:14.374094 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-11044ae9-5b55-44ff-8d70-25fba1b363f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:07:14.386106 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11044ae9-5b55-44ff-8d70-25fba1b363f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:07:14.387700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11044ae9-5b55-44ff-8d70-25fba1b363f9 service placement] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:07:14.398624 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1189/2411] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:07:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:07:24.394266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=32 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:08:54.735715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a650e-b8d7-4674-9b8a-2fe7086b9e75 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:54.835127 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a650e-b8d7-4674-9b8a-2fe7086b9e75 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:54.840395 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953a302-b9df-4ceb-a946-edaf05094235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:54.841078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953a302-b9df-4ceb-a946-edaf05094235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:54.841356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953a302-b9df-4ceb-a946-edaf05094235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:54.841631 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953a302-b9df-4ceb-a946-edaf05094235 admin admin] PATH_INFO: `/v3/users/1678f28de9e04a80bab033fd76fcd8b7` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:54.842904 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0953a302-b9df-4ceb-a946-edaf05094235 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1678f28de9e04a80bab033fd76fcd8b7)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:54.844927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0953a302-b9df-4ceb-a946-edaf05094235 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:54.897996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0953a302-b9df-4ceb-a946-edaf05094235 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1678f28de9e04a80bab033fd76fcd8b7'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:54.898660 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0953a302-b9df-4ceb-a946-edaf05094235 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1678f28de9e04a80bab033fd76fcd8b7'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:54.909581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0953a302-b9df-4ceb-a946-edaf05094235 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1678f28de9e04a80bab033fd76fcd8b7'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:54.925664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0953a302-b9df-4ceb-a946-edaf05094235 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1678f28de9e04a80bab033fd76fcd8b7'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:54.932934 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0953a302-b9df-4ceb-a946-edaf05094235 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1678f28de9e04a80bab033fd76fcd8b7'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:54.940887 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1223/2412] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:08:54 2022] DELETE /identity/v3/users/1678f28de9e04a80bab033fd76fcd8b7 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:08:54.946052 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7000fb2-9f1e-45dd-8ee2-b113ff82cabe None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:54.946605 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7000fb2-9f1e-45dd-8ee2-b113ff82cabe None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:54.946605 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7000fb2-9f1e-45dd-8ee2-b113ff82cabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:54.979493 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a7000fb2-9f1e-45dd-8ee2-b113ff82cabe None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:08:54.980946 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a7000fb2-9f1e-45dd-8ee2-b113ff82cabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:55.076661 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1190/2413] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:08:55.090077 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953a302-b9df-4ceb-a946-edaf05094235 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:55.093290 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0953a302-b9df-4ceb-a946-edaf05094235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:55.175366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953a302-b9df-4ceb-a946-edaf05094235 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:55.181494 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f3166-a092-43e7-9bd5-d0a87a988248 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:55.182005 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f3166-a092-43e7-9bd5-d0a87a988248 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:55.182362 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f3166-a092-43e7-9bd5-d0a87a988248 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:55.182682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f3166-a092-43e7-9bd5-d0a87a988248 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:55.197151 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f3166-a092-43e7-9bd5-d0a87a988248 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:55.201012 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f3166-a092-43e7-9bd5-d0a87a988248 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:55.219412 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1224/2414] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:08:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:08:55.516811 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7000fb2-9f1e-45dd-8ee2-b113ff82cabe None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:55.519146 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a7000fb2-9f1e-45dd-8ee2-b113ff82cabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:55.533188 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7000fb2-9f1e-45dd-8ee2-b113ff82cabe None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:55.538487 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6cdb8-a770-4b97-9383-a8dcb172822a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:55.539156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6cdb8-a770-4b97-9383-a8dcb172822a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:55.539439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6cdb8-a770-4b97-9383-a8dcb172822a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:55.539715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6cdb8-a770-4b97-9383-a8dcb172822a admin admin] PATH_INFO: `/v3/projects/07e6cfce1a9a480fa039a0158683103f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:55.541471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6cdb8-a770-4b97-9383-a8dcb172822a admin admin] RBAC: Authorizing `identity:delete_project(project_id=07e6cfce1a9a480fa039a0158683103f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:55.543475 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6cdb8-a770-4b97-9383-a8dcb172822a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:55.618713 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-71f6cdb8-a770-4b97-9383-a8dcb172822a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07e6cfce1a9a480fa039a0158683103f', 'request_id': 'req-71f6cdb8-a770-4b97-9383-a8dcb172822a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:55.627792 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-71f6cdb8-a770-4b97-9383-a8dcb172822a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07e6cfce1a9a480fa039a0158683103f', 'request_id': 'req-71f6cdb8-a770-4b97-9383-a8dcb172822a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:55.636858 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-71f6cdb8-a770-4b97-9383-a8dcb172822a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07e6cfce1a9a480fa039a0158683103f', 'request_id': 'req-71f6cdb8-a770-4b97-9383-a8dcb172822a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:55.679516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-71f6cdb8-a770-4b97-9383-a8dcb172822a admin admin] The token cache is being invalidate because project 07e6cfce1a9a480fa039a0158683103f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:08:55.680303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-71f6cdb8-a770-4b97-9383-a8dcb172822a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:55.681332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-71f6cdb8-a770-4b97-9383-a8dcb172822a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:55.683292 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1191/2415] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:08:55 2022] DELETE /identity/v3/projects/07e6cfce1a9a480fa039a0158683103f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:08:55.687720 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f3166-a092-43e7-9bd5-d0a87a988248 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:55.700208 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f3166-a092-43e7-9bd5-d0a87a988248 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:55.732176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:08:55.787614 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90eda22-18f8-4a33-97d1-74ff2e1182de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:55.788273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90eda22-18f8-4a33-97d1-74ff2e1182de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:55.788845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90eda22-18f8-4a33-97d1-74ff2e1182de admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:55.789097 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90eda22-18f8-4a33-97d1-74ff2e1182de admin admin] PATH_INFO: `/v3/users/1ff6a9c6a2174a37a0f2bafba55dc987` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:55.791080 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90eda22-18f8-4a33-97d1-74ff2e1182de admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ff6a9c6a2174a37a0f2bafba55dc987)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:55.794332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90eda22-18f8-4a33-97d1-74ff2e1182de admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:55.894273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e90eda22-18f8-4a33-97d1-74ff2e1182de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ff6a9c6a2174a37a0f2bafba55dc987'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:55.895023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e90eda22-18f8-4a33-97d1-74ff2e1182de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ff6a9c6a2174a37a0f2bafba55dc987'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:55.908286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e90eda22-18f8-4a33-97d1-74ff2e1182de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ff6a9c6a2174a37a0f2bafba55dc987'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:55.933533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e90eda22-18f8-4a33-97d1-74ff2e1182de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ff6a9c6a2174a37a0f2bafba55dc987'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:55.942363 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e90eda22-18f8-4a33-97d1-74ff2e1182de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ff6a9c6a2174a37a0f2bafba55dc987'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:55.950402 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1225/2416] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:08:55 2022] DELETE /identity/v3/users/1ff6a9c6a2174a37a0f2bafba55dc987 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:08:56.450344 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6cdb8-a770-4b97-9383-a8dcb172822a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:56.453512 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-71f6cdb8-a770-4b97-9383-a8dcb172822a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:56.474752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6cdb8-a770-4b97-9383-a8dcb172822a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:56.553997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90eda22-18f8-4a33-97d1-74ff2e1182de admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:56.555527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e90eda22-18f8-4a33-97d1-74ff2e1182de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:56.598326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a45310-04ff-4c2f-b3b4-8bebf8a34fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:56.600198 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a45310-04ff-4c2f-b3b4-8bebf8a34fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:56.600723 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a45310-04ff-4c2f-b3b4-8bebf8a34fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:56.601189 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a45310-04ff-4c2f-b3b4-8bebf8a34fa4 admin admin] PATH_INFO: `/v3/projects/9636fc5cb75b421496fdd49fc2779edb` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:56.602698 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a45310-04ff-4c2f-b3b4-8bebf8a34fa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9636fc5cb75b421496fdd49fc2779edb)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:56.605406 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a45310-04ff-4c2f-b3b4-8bebf8a34fa4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:56.630507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:08:56.669233 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90eda22-18f8-4a33-97d1-74ff2e1182de admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:56.688960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-01a45310-04ff-4c2f-b3b4-8bebf8a34fa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9636fc5cb75b421496fdd49fc2779edb', 'request_id': 'req-01a45310-04ff-4c2f-b3b4-8bebf8a34fa4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:56.697249 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-01a45310-04ff-4c2f-b3b4-8bebf8a34fa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9636fc5cb75b421496fdd49fc2779edb', 'request_id': 'req-01a45310-04ff-4c2f-b3b4-8bebf8a34fa4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:56.704308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-01a45310-04ff-4c2f-b3b4-8bebf8a34fa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9636fc5cb75b421496fdd49fc2779edb', 'request_id': 'req-01a45310-04ff-4c2f-b3b4-8bebf8a34fa4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:56.720058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:08:56.736155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-01a45310-04ff-4c2f-b3b4-8bebf8a34fa4 admin admin] The token cache is being invalidate because project 9636fc5cb75b421496fdd49fc2779edb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:08:56.738049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-01a45310-04ff-4c2f-b3b4-8bebf8a34fa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:56.738346 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-01a45310-04ff-4c2f-b3b4-8bebf8a34fa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:56.743182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1192/2417] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:08:56 2022] DELETE /identity/v3/projects/9636fc5cb75b421496fdd49fc2779edb => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:08:56.748041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094187dc-1c59-48e1-917c-749c34dce9c4 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:56.748329 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094187dc-1c59-48e1-917c-749c34dce9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:56.748607 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094187dc-1c59-48e1-917c-749c34dce9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:56.780225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66385a-8ed9-441d-be4b-bf9ef4858e2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:56.780649 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66385a-8ed9-441d-be4b-bf9ef4858e2c service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:56.780936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66385a-8ed9-441d-be4b-bf9ef4858e2c service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:56.781216 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66385a-8ed9-441d-be4b-bf9ef4858e2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:56.782844 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-094187dc-1c59-48e1-917c-749c34dce9c4 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:08:56.783131 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8e66385a-8ed9-441d-be4b-bf9ef4858e2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:56.784479 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-094187dc-1c59-48e1-917c-749c34dce9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:56.797261 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66385a-8ed9-441d-be4b-bf9ef4858e2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:56.799021 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66385a-8ed9-441d-be4b-bf9ef4858e2c service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:56.812971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1226/2418] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:08:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:08:56.881782 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1193/2419] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:08:56.890405 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae082c22-9467-4a07-97de-5d9ac3326156 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:56.891039 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae082c22-9467-4a07-97de-5d9ac3326156 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:56.891530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae082c22-9467-4a07-97de-5d9ac3326156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:56.917230 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-ae082c22-9467-4a07-97de-5d9ac3326156 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:08:56.918520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ae082c22-9467-4a07-97de-5d9ac3326156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:56.927372 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1227/2420] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:08:56.931450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094187dc-1c59-48e1-917c-749c34dce9c4 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:56.943615 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094187dc-1c59-48e1-917c-749c34dce9c4 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:56.948486 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140c73c-2a54-4e79-89ee-4e94fc2b49f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:56.949272 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140c73c-2a54-4e79-89ee-4e94fc2b49f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:56.949560 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140c73c-2a54-4e79-89ee-4e94fc2b49f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:56.949838 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140c73c-2a54-4e79-89ee-4e94fc2b49f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:56.951271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140c73c-2a54-4e79-89ee-4e94fc2b49f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:08:56.951271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140c73c-2a54-4e79-89ee-4e94fc2b49f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:56.962431 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140c73c-2a54-4e79-89ee-4e94fc2b49f5 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:56.993446 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1194/2421] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:08:56.998045 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae082c22-9467-4a07-97de-5d9ac3326156 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:57.010039 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae082c22-9467-4a07-97de-5d9ac3326156 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:57.014851 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e5942-9028-4f51-b101-10892c2f893d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:57.015813 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e5942-9028-4f51-b101-10892c2f893d admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:57.016303 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e5942-9028-4f51-b101-10892c2f893d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:57.016812 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e5942-9028-4f51-b101-10892c2f893d admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:57.017577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e5942-9028-4f51-b101-10892c2f893d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:57.020166 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e5942-9028-4f51-b101-10892c2f893d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:57.048984 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140c73c-2a54-4e79-89ee-4e94fc2b49f5 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:57.054917 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f140c73c-2a54-4e79-89ee-4e94fc2b49f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:57.065481 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1228/2422] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:08:56 2022] POST /identity/v3/projects => generated 393 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:08:57.072464 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e5942-9028-4f51-b101-10892c2f893d admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:57.088595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140c73c-2a54-4e79-89ee-4e94fc2b49f5 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:57.199102 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e5942-9028-4f51-b101-10892c2f893d admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:57.222213 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b2caf-e0c4-444a-a108-8d11f22feef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:57.223273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b2caf-e0c4-444a-a108-8d11f22feef2 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:57.223273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b2caf-e0c4-444a-a108-8d11f22feef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:57.223492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b2caf-e0c4-444a-a108-8d11f22feef2 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:57.224061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b2caf-e0c4-444a-a108-8d11f22feef2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:57.226270 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b2caf-e0c4-444a-a108-8d11f22feef2 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:57.234221 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9bcf8-c113-46fe-ae07-cfce65f4d57d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:57.234954 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9bcf8-c113-46fe-ae07-cfce65f4d57d service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:57.235414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9bcf8-c113-46fe-ae07-cfce65f4d57d service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:57.235704 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9bcf8-c113-46fe-ae07-cfce65f4d57d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:57.238659 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c2d9bcf8-c113-46fe-ae07-cfce65f4d57d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:57.254660 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9bcf8-c113-46fe-ae07-cfce65f4d57d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:57.256709 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9bcf8-c113-46fe-ae07-cfce65f4d57d service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:57.266790 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1229/2423] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:08:57 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:08:57.273527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b2caf-e0c4-444a-a108-8d11f22feef2 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:57.287887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1195/2424] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:08:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:08:57.300856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b2caf-e0c4-444a-a108-8d11f22feef2 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:57.311364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2a833-6811-4eac-aa96-e7aec6544103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:57.311364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2a833-6811-4eac-aa96-e7aec6544103 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:57.314007 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2a833-6811-4eac-aa96-e7aec6544103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:57.314007 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2a833-6811-4eac-aa96-e7aec6544103 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:57.314007 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2a833-6811-4eac-aa96-e7aec6544103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:57.318277 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2a833-6811-4eac-aa96-e7aec6544103 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:57.327426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9bcf8-c113-46fe-ae07-cfce65f4d57d service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:57.329497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c2d9bcf8-c113-46fe-ae07-cfce65f4d57d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:57.333234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1230/2425] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:08:57.334906 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2a833-6811-4eac-aa96-e7aec6544103 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:57.356470 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2a833-6811-4eac-aa96-e7aec6544103 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:57.358012 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e1611-dc6d-4d3e-8ece-058cc63917c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:57.359065 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e1611-dc6d-4d3e-8ece-058cc63917c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:57.359557 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e1611-dc6d-4d3e-8ece-058cc63917c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:57.360005 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e1611-dc6d-4d3e-8ece-058cc63917c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:57.360785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e1611-dc6d-4d3e-8ece-058cc63917c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:57.363339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e1611-dc6d-4d3e-8ece-058cc63917c3 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:57.377121 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:08:57.377279 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1231/2426] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:08:57.384257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e1611-dc6d-4d3e-8ece-058cc63917c3 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:57.397225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e1611-dc6d-4d3e-8ece-058cc63917c3 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:57.404830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6146e-8c5b-4c3e-99a3-411f10a6fd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:57.406149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6146e-8c5b-4c3e-99a3-411f10a6fd2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:57.406655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6146e-8c5b-4c3e-99a3-411f10a6fd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:57.407134 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6146e-8c5b-4c3e-99a3-411f10a6fd2a admin admin] PATH_INFO: `/v3/projects/0e4796a5773e4db48511af809d52da6c/users/f11853e48a7a4229841ee9c711c7bea5/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:57.431155 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6146e-8c5b-4c3e-99a3-411f10a6fd2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e4796a5773e4db48511af809d52da6c, user_id=f11853e48a7a4229841ee9c711c7bea5, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:57.433748 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6146e-8c5b-4c3e-99a3-411f10a6fd2a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:57.440689 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9bcf8-c113-46fe-ae07-cfce65f4d57d service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:57.445514 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86c2af-b75b-4d3c-a85b-ea762af5c646 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:57.446233 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86c2af-b75b-4d3c-a85b-ea762af5c646 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:57.446579 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86c2af-b75b-4d3c-a85b-ea762af5c646 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:57.446814 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86c2af-b75b-4d3c-a85b-ea762af5c646 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:57.449784 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bf86c2af-b75b-4d3c-a85b-ea762af5c646 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:57.462911 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1232/2427] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:08:57 2022] PUT /identity/v3/projects/0e4796a5773e4db48511af809d52da6c/users/f11853e48a7a4229841ee9c711c7bea5/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:08:57.462911 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6146e-8c5b-4c3e-99a3-411f10a6fd2a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:57.462911 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d6f6146e-8c5b-4c3e-99a3-411f10a6fd2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:57.463932 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86c2af-b75b-4d3c-a85b-ea762af5c646 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:57.465427 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86c2af-b75b-4d3c-a85b-ea762af5c646 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:57.476770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1196/2428] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:08:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:08:57.477732 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85e69b-418c-41c6-992c-56233145673a None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:57.478065 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85e69b-418c-41c6-992c-56233145673a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:57.478355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85e69b-418c-41c6-992c-56233145673a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:57.505742 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:08:57.521362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-2e85e69b-418c-41c6-992c-56233145673a None None] MFA Rules not processed for user `f11853e48a7a4229841ee9c711c7bea5`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:08:57.523340 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2e85e69b-418c-41c6-992c-56233145673a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:57.552862 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6146e-8c5b-4c3e-99a3-411f10a6fd2a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:57.560744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753dc76-7810-490e-aa70-330272552a41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:57.561535 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753dc76-7810-490e-aa70-330272552a41 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:57.562333 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753dc76-7810-490e-aa70-330272552a41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:57.562646 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753dc76-7810-490e-aa70-330272552a41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:57.565673 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a753dc76-7810-490e-aa70-330272552a41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:57.653364 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1197/2429] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:08:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:08:57.659368 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85e69b-418c-41c6-992c-56233145673a None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:57.680834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753dc76-7810-490e-aa70-330272552a41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:57.684153 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753dc76-7810-490e-aa70-330272552a41 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:57.713991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1233/2430] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:08:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:08:57.754485 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85e69b-418c-41c6-992c-56233145673a None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:57.759964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfd1da-3e4d-4338-b140-cc2fca566377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:57.760608 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfd1da-3e4d-4338-b140-cc2fca566377 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:57.760898 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfd1da-3e4d-4338-b140-cc2fca566377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:57.761178 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfd1da-3e4d-4338-b140-cc2fca566377 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:57.761854 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cfd1da-3e4d-4338-b140-cc2fca566377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:57.763924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cfd1da-3e4d-4338-b140-cc2fca566377 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:57.795936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1198/2431] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:08:57 2022] POST /identity/v3/projects => generated 393 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:08:57.802540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753dc76-7810-490e-aa70-330272552a41 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:57.820586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753dc76-7810-490e-aa70-330272552a41 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:57.917183 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f13886-6de5-467a-a86e-6d885a0617a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:57.918083 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f13886-6de5-467a-a86e-6d885a0617a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:57.918454 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f13886-6de5-467a-a86e-6d885a0617a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:57.918854 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f13886-6de5-467a-a86e-6d885a0617a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:57.919909 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f13886-6de5-467a-a86e-6d885a0617a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:57.922203 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f13886-6de5-467a-a86e-6d885a0617a5 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:57.950047 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1234/2432] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:08:57 2022] POST /identity/v3/users => generated 303 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:08:57.951781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfd1da-3e4d-4338-b140-cc2fca566377 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:57.965699 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfd1da-3e4d-4338-b140-cc2fca566377 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:57.974254 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62a018-e1f4-473e-b002-07befccac9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:57.975137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62a018-e1f4-473e-b002-07befccac9bd admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:57.975269 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62a018-e1f4-473e-b002-07befccac9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:57.975456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62a018-e1f4-473e-b002-07befccac9bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:57.976170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab62a018-e1f4-473e-b002-07befccac9bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:57.977809 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab62a018-e1f4-473e-b002-07befccac9bd admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:57.989952 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1199/2433] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:08:57.994677 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f13886-6de5-467a-a86e-6d885a0617a5 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:58.006107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f13886-6de5-467a-a86e-6d885a0617a5 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:58.011429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a1194-e7f8-484c-a4b0-9944eb764fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:58.012125 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a1194-e7f8-484c-a4b0-9944eb764fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:58.012682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a1194-e7f8-484c-a4b0-9944eb764fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:58.012963 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a1194-e7f8-484c-a4b0-9944eb764fb1 admin admin] PATH_INFO: `/v3/projects/82e81dc435d94497b2559b2b41d31baa/users/4ebc4a4b21dd4c6cb847c1ad751b105b/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:58.038330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a1194-e7f8-484c-a4b0-9944eb764fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82e81dc435d94497b2559b2b41d31baa, user_id=4ebc4a4b21dd4c6cb847c1ad751b105b, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:58.046359 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a1194-e7f8-484c-a4b0-9944eb764fb1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:58.071997 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1235/2434] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:08:57 2022] PUT /identity/v3/projects/82e81dc435d94497b2559b2b41d31baa/users/4ebc4a4b21dd4c6cb847c1ad751b105b/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:08:58.075690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62a018-e1f4-473e-b002-07befccac9bd admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:58.088142 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62a018-e1f4-473e-b002-07befccac9bd admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:58.119245 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:08:58.165000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc81ae8-dac9-46a0-b6a8-114642bff722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:58.166208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc81ae8-dac9-46a0-b6a8-114642bff722 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:58.166656 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc81ae8-dac9-46a0-b6a8-114642bff722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:58.167132 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc81ae8-dac9-46a0-b6a8-114642bff722 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:58.168004 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc81ae8-dac9-46a0-b6a8-114642bff722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:58.169825 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc81ae8-dac9-46a0-b6a8-114642bff722 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:58.182674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1200/2435] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:08:58.187066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a1194-e7f8-484c-a4b0-9944eb764fb1 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:58.199488 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a1194-e7f8-484c-a4b0-9944eb764fb1 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:58.208807 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fd2ba-6fd5-4d0c-91d3-21cd4e5e59e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:58.209661 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fd2ba-6fd5-4d0c-91d3-21cd4e5e59e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:58.209851 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fd2ba-6fd5-4d0c-91d3-21cd4e5e59e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:58.210245 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fd2ba-6fd5-4d0c-91d3-21cd4e5e59e2 admin admin] PATH_INFO: `/v3/projects/82e81dc435d94497b2559b2b41d31baa/users/4ebc4a4b21dd4c6cb847c1ad751b105b/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:58.212859 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684fd2ba-6fd5-4d0c-91d3-21cd4e5e59e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82e81dc435d94497b2559b2b41d31baa, user_id=4ebc4a4b21dd4c6cb847c1ad751b105b, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:58.215826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684fd2ba-6fd5-4d0c-91d3-21cd4e5e59e2 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:58.229269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1236/2436] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:08:58 2022] PUT /identity/v3/projects/82e81dc435d94497b2559b2b41d31baa/users/4ebc4a4b21dd4c6cb847c1ad751b105b/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:08:58.241328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca8a2e-9dd1-486d-bc3c-9204cdbe2af4 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:58.241645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca8a2e-9dd1-486d-bc3c-9204cdbe2af4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:58.241925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca8a2e-9dd1-486d-bc3c-9204cdbe2af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:58.276653 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a7ca8a2e-9dd1-486d-bc3c-9204cdbe2af4 None None] MFA Rules not processed for user `4ebc4a4b21dd4c6cb847c1ad751b105b`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:08:58.277732 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a7ca8a2e-9dd1-486d-bc3c-9204cdbe2af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:58.384685 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1201/2437] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:08:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:08:58.389597 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fd2ba-6fd5-4d0c-91d3-21cd4e5e59e2 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:58.392008 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-684fd2ba-6fd5-4d0c-91d3-21cd4e5e59e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:58.398873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca8a2e-9dd1-486d-bc3c-9204cdbe2af4 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:58.456892 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:08:58.495871 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fd2ba-6fd5-4d0c-91d3-21cd4e5e59e2 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:58.499284 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca8a2e-9dd1-486d-bc3c-9204cdbe2af4 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:58.500195 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f8288-03b9-4e3e-b8b7-e13841649db6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:58.500934 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f8288-03b9-4e3e-b8b7-e13841649db6 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:58.501247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f8288-03b9-4e3e-b8b7-e13841649db6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:58.501588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f8288-03b9-4e3e-b8b7-e13841649db6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:58.503402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f45c9-0467-4ca5-9ca7-6d55ab90dfc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:58.504905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f45c9-0467-4ca5-9ca7-6d55ab90dfc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:58.505119 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f45c9-0467-4ca5-9ca7-6d55ab90dfc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:58.505407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f45c9-0467-4ca5-9ca7-6d55ab90dfc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:58.507593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-874f45c9-0467-4ca5-9ca7-6d55ab90dfc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:58.512668 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f8288-03b9-4e3e-b8b7-e13841649db6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:58.514661 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f8288-03b9-4e3e-b8b7-e13841649db6 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:58.545104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1237/2438] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:08:58 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:08:58.609293 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f8288-03b9-4e3e-b8b7-e13841649db6 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:58.612452 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-dc4f8288-03b9-4e3e-b8b7-e13841649db6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:58.626925 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f8288-03b9-4e3e-b8b7-e13841649db6 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:58.631701 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f45c9-0467-4ca5-9ca7-6d55ab90dfc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:58.633773 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f45c9-0467-4ca5-9ca7-6d55ab90dfc0 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:58.636733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf96b76-722d-4a6b-a06b-b590e00f55fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:58.637709 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf96b76-722d-4a6b-a06b-b590e00f55fa service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:58.638546 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:08:58.644184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf96b76-722d-4a6b-a06b-b590e00f55fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:58.644611 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf96b76-722d-4a6b-a06b-b590e00f55fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:58.647821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1202/2439] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:08:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:08:58.731905 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf96b76-722d-4a6b-a06b-b590e00f55fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:58.733394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf96b76-722d-4a6b-a06b-b590e00f55fa service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:58.746492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1238/2440] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:08:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:08:59.615951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f45c9-0467-4ca5-9ca7-6d55ab90dfc0 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:59.619280 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-874f45c9-0467-4ca5-9ca7-6d55ab90dfc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:59.705369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf96b76-722d-4a6b-a06b-b590e00f55fa service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:08:59.707672 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-eaf96b76-722d-4a6b-a06b-b590e00f55fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:59.739194 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf96b76-722d-4a6b-a06b-b590e00f55fa service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:59.751934 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcff0a0-ad5f-4f98-93be-3984c142533a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:59.754733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcff0a0-ad5f-4f98-93be-3984c142533a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:59.755399 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcff0a0-ad5f-4f98-93be-3984c142533a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:59.756546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcff0a0-ad5f-4f98-93be-3984c142533a admin admin] PATH_INFO: `/v3/users/77cb30fae1184064a8c4f822d4d31096` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:59.759010 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcff0a0-ad5f-4f98-93be-3984c142533a admin admin] RBAC: Authorizing `identity:delete_user(user_id=77cb30fae1184064a8c4f822d4d31096)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:59.764005 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcff0a0-ad5f-4f98-93be-3984c142533a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:59.783667 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f45c9-0467-4ca5-9ca7-6d55ab90dfc0 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:08:59.803079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e999e-e704-43ba-b24b-5599fdd4e0b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:08:59.804237 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e999e-e704-43ba-b24b-5599fdd4e0b5 service placement] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:08:59.806040 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e999e-e704-43ba-b24b-5599fdd4e0b5 service placement] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:08:59.807485 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e999e-e704-43ba-b24b-5599fdd4e0b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:08:59.817083 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f32e999e-e704-43ba-b24b-5599fdd4e0b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:08:59.853479 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e999e-e704-43ba-b24b-5599fdd4e0b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:08:59.856939 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e999e-e704-43ba-b24b-5599fdd4e0b5 service placement] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:08:59.893853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1203/2441] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:08:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:08:59.906657 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5bcff0a0-ad5f-4f98-93be-3984c142533a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77cb30fae1184064a8c4f822d4d31096'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:59.910323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5bcff0a0-ad5f-4f98-93be-3984c142533a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77cb30fae1184064a8c4f822d4d31096'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:59.938478 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5bcff0a0-ad5f-4f98-93be-3984c142533a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77cb30fae1184064a8c4f822d4d31096'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:59.973474 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5bcff0a0-ad5f-4f98-93be-3984c142533a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77cb30fae1184064a8c4f822d4d31096'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:59.990523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5bcff0a0-ad5f-4f98-93be-3984c142533a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77cb30fae1184064a8c4f822d4d31096'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:08:59.999323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1239/2442] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:08:59 2022] DELETE /identity/v3/users/77cb30fae1184064a8c4f822d4d31096 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:00.601365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e999e-e704-43ba-b24b-5599fdd4e0b5 service placement] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:00.629456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e999e-e704-43ba-b24b-5599fdd4e0b5 service placement] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:00.779252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c550cb-c069-4e43-bcd1-a0a9578e642b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:00.779942 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c550cb-c069-4e43-bcd1-a0a9578e642b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:00.780226 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c550cb-c069-4e43-bcd1-a0a9578e642b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:00.780503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c550cb-c069-4e43-bcd1-a0a9578e642b admin admin] PATH_INFO: `/v3/projects/38f80d210cce4a41af527ac134deba93` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:00.782714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c550cb-c069-4e43-bcd1-a0a9578e642b admin admin] RBAC: Authorizing `identity:delete_project(project_id=38f80d210cce4a41af527ac134deba93)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:00.788452 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c550cb-c069-4e43-bcd1-a0a9578e642b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:00.947383 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-91c550cb-c069-4e43-bcd1-a0a9578e642b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38f80d210cce4a41af527ac134deba93', 'request_id': 'req-91c550cb-c069-4e43-bcd1-a0a9578e642b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:00.961716 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-91c550cb-c069-4e43-bcd1-a0a9578e642b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38f80d210cce4a41af527ac134deba93', 'request_id': 'req-91c550cb-c069-4e43-bcd1-a0a9578e642b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:00.997450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-91c550cb-c069-4e43-bcd1-a0a9578e642b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38f80d210cce4a41af527ac134deba93', 'request_id': 'req-91c550cb-c069-4e43-bcd1-a0a9578e642b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:01.023660 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-91c550cb-c069-4e43-bcd1-a0a9578e642b admin admin] The token cache is being invalidate because project 38f80d210cce4a41af527ac134deba93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:01.023958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-91c550cb-c069-4e43-bcd1-a0a9578e642b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:01.026335 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-91c550cb-c069-4e43-bcd1-a0a9578e642b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:01.030052 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1204/2443] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:00 2022] DELETE /identity/v3/projects/38f80d210cce4a41af527ac134deba93 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:02.434383 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcff0a0-ad5f-4f98-93be-3984c142533a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:02.435947 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5bcff0a0-ad5f-4f98-93be-3984c142533a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:02.521100 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:02.548672 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c550cb-c069-4e43-bcd1-a0a9578e642b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:02.550200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-91c550cb-c069-4e43-bcd1-a0a9578e642b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:02.574623 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcff0a0-ad5f-4f98-93be-3984c142533a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:02.591762 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:02.668644 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbd0d4-2825-4f25-b339-c7985487bcdf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:02.669691 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbd0d4-2825-4f25-b339-c7985487bcdf service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:02.670197 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbd0d4-2825-4f25-b339-c7985487bcdf service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:02.670664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbd0d4-2825-4f25-b339-c7985487bcdf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:02.675308 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7bdbd0d4-2825-4f25-b339-c7985487bcdf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:02.690717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdbd0d4-2825-4f25-b339-c7985487bcdf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:02.691443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c550cb-c069-4e43-bcd1-a0a9578e642b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:02.694380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdbd0d4-2825-4f25-b339-c7985487bcdf service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:02.699935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a1f55-7b4a-4ff6-851b-3d0c85c77bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:02.700946 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a1f55-7b4a-4ff6-851b-3d0c85c77bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:02.701846 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a1f55-7b4a-4ff6-851b-3d0c85c77bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:02.702512 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a1f55-7b4a-4ff6-851b-3d0c85c77bd2 admin admin] PATH_INFO: `/v3/users/4001e8ec57b34f57aae1654fad9217f8` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:02.709714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a1f55-7b4a-4ff6-851b-3d0c85c77bd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4001e8ec57b34f57aae1654fad9217f8)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:02.713663 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a1f55-7b4a-4ff6-851b-3d0c85c77bd2 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:02.715548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1240/2444] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:02.809705 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-956a1f55-7b4a-4ff6-851b-3d0c85c77bd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4001e8ec57b34f57aae1654fad9217f8'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:02.810676 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-956a1f55-7b4a-4ff6-851b-3d0c85c77bd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4001e8ec57b34f57aae1654fad9217f8'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:02.830815 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-956a1f55-7b4a-4ff6-851b-3d0c85c77bd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4001e8ec57b34f57aae1654fad9217f8'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:02.860498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-956a1f55-7b4a-4ff6-851b-3d0c85c77bd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4001e8ec57b34f57aae1654fad9217f8'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:02.861801 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-956a1f55-7b4a-4ff6-851b-3d0c85c77bd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4001e8ec57b34f57aae1654fad9217f8'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:02.879685 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1205/2445] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:02 2022] DELETE /identity/v3/users/4001e8ec57b34f57aae1654fad9217f8 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:02.927784 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbd0d4-2825-4f25-b339-c7985487bcdf service placement] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:02.929323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7bdbd0d4-2825-4f25-b339-c7985487bcdf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:02.949914 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbd0d4-2825-4f25-b339-c7985487bcdf service placement] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:03.043707 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72001668-fb8b-493a-b7d4-fa65d0a31d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:03.044617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72001668-fb8b-493a-b7d4-fa65d0a31d03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:03.044958 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72001668-fb8b-493a-b7d4-fa65d0a31d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:03.045419 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72001668-fb8b-493a-b7d4-fa65d0a31d03 admin admin] PATH_INFO: `/v3/users/7d1b6c388c7642a892c905d16f3b8d50` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:03.047631 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72001668-fb8b-493a-b7d4-fa65d0a31d03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d1b6c388c7642a892c905d16f3b8d50)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:03.048862 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72001668-fb8b-493a-b7d4-fa65d0a31d03 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:03.135945 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-72001668-fb8b-493a-b7d4-fa65d0a31d03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d1b6c388c7642a892c905d16f3b8d50'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:03.136774 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-72001668-fb8b-493a-b7d4-fa65d0a31d03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d1b6c388c7642a892c905d16f3b8d50'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:03.151808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-72001668-fb8b-493a-b7d4-fa65d0a31d03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d1b6c388c7642a892c905d16f3b8d50'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:03.155099 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a1f55-7b4a-4ff6-851b-3d0c85c77bd2 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:03.156678 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-956a1f55-7b4a-4ff6-851b-3d0c85c77bd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:03.183688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-72001668-fb8b-493a-b7d4-fa65d0a31d03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d1b6c388c7642a892c905d16f3b8d50'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:03.184343 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a1f55-7b4a-4ff6-851b-3d0c85c77bd2 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:03.195647 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd5dd0-c604-4af3-9379-c1ff68ed60a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:03.196939 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:03.197295 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd5dd0-c604-4af3-9379-c1ff68ed60a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:03.198776 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd5dd0-c604-4af3-9379-c1ff68ed60a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:03.199180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd5dd0-c604-4af3-9379-c1ff68ed60a4 admin admin] PATH_INFO: `/v3/projects/7ded351d7a1b4a9c8bd4901f8becfe5b` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:03.201233 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd5dd0-c604-4af3-9379-c1ff68ed60a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ded351d7a1b4a9c8bd4901f8becfe5b)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:03.203029 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-72001668-fb8b-493a-b7d4-fa65d0a31d03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d1b6c388c7642a892c905d16f3b8d50'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:03.208952 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd5dd0-c604-4af3-9379-c1ff68ed60a4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:03.218822 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1241/2446] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:02 2022] DELETE /identity/v3/users/7d1b6c388c7642a892c905d16f3b8d50 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:03.283955 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-80bd5dd0-c604-4af3-9379-c1ff68ed60a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ded351d7a1b4a9c8bd4901f8becfe5b', 'request_id': 'req-80bd5dd0-c604-4af3-9379-c1ff68ed60a4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:03.299002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-80bd5dd0-c604-4af3-9379-c1ff68ed60a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ded351d7a1b4a9c8bd4901f8becfe5b', 'request_id': 'req-80bd5dd0-c604-4af3-9379-c1ff68ed60a4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:03.310728 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-80bd5dd0-c604-4af3-9379-c1ff68ed60a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ded351d7a1b4a9c8bd4901f8becfe5b', 'request_id': 'req-80bd5dd0-c604-4af3-9379-c1ff68ed60a4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:03.338191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-80bd5dd0-c604-4af3-9379-c1ff68ed60a4 admin admin] The token cache is being invalidate because project 7ded351d7a1b4a9c8bd4901f8becfe5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:03.339726 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-80bd5dd0-c604-4af3-9379-c1ff68ed60a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:03.341719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-80bd5dd0-c604-4af3-9379-c1ff68ed60a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:03.350660 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1206/2447] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:03 2022] DELETE /identity/v3/projects/7ded351d7a1b4a9c8bd4901f8becfe5b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:03.352875 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58affdd3-21d2-431b-9559-558fd59c1f86 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:03.353175 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58affdd3-21d2-431b-9559-558fd59c1f86 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:03.353427 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58affdd3-21d2-431b-9559-558fd59c1f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:03.383634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-58affdd3-21d2-431b-9559-558fd59c1f86 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:03.385168 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-58affdd3-21d2-431b-9559-558fd59c1f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:03.402400 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd5dd0-c604-4af3-9379-c1ff68ed60a4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:03.404040 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-80bd5dd0-c604-4af3-9379-c1ff68ed60a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:03.433101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:03.475829 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1242/2448] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:03.485863 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7a5fc-1716-410b-aae5-ea3575a90709 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:03.486265 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7a5fc-1716-410b-aae5-ea3575a90709 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:03.487046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7a5fc-1716-410b-aae5-ea3575a90709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:03.495645 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd5dd0-c604-4af3-9379-c1ff68ed60a4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:03.504705 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55599cf3-f79f-49ee-928a-ca8c555bd979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:03.505853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55599cf3-f79f-49ee-928a-ca8c555bd979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:03.506557 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55599cf3-f79f-49ee-928a-ca8c555bd979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:03.506898 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55599cf3-f79f-49ee-928a-ca8c555bd979 admin admin] PATH_INFO: `/v3/users/f11853e48a7a4229841ee9c711c7bea5` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:03.509016 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55599cf3-f79f-49ee-928a-ca8c555bd979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f11853e48a7a4229841ee9c711c7bea5)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:03.512912 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55599cf3-f79f-49ee-928a-ca8c555bd979 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:03.518409 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-b8b7a5fc-1716-410b-aae5-ea3575a90709 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:03.519984 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b8b7a5fc-1716-410b-aae5-ea3575a90709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:03.532889 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1243/2449] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:03.533632 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7a5fc-1716-410b-aae5-ea3575a90709 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:03.535006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b8b7a5fc-1716-410b-aae5-ea3575a90709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:03.552123 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7a5fc-1716-410b-aae5-ea3575a90709 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:03.557727 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9335ef6-2bcf-42f8-9f65-e0858b352185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:03.559101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9335ef6-2bcf-42f8-9f65-e0858b352185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:03.559603 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9335ef6-2bcf-42f8-9f65-e0858b352185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:03.560027 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9335ef6-2bcf-42f8-9f65-e0858b352185 admin admin] PATH_INFO: `/v3/projects/e829698141514a009e2e769854fe66f0` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:03.561739 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9335ef6-2bcf-42f8-9f65-e0858b352185 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e829698141514a009e2e769854fe66f0)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:03.563812 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9335ef6-2bcf-42f8-9f65-e0858b352185 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:03.577829 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-55599cf3-f79f-49ee-928a-ca8c555bd979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f11853e48a7a4229841ee9c711c7bea5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:03.578779 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-55599cf3-f79f-49ee-928a-ca8c555bd979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f11853e48a7a4229841ee9c711c7bea5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:03.588067 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-55599cf3-f79f-49ee-928a-ca8c555bd979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f11853e48a7a4229841ee9c711c7bea5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:03.603764 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-55599cf3-f79f-49ee-928a-ca8c555bd979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f11853e48a7a4229841ee9c711c7bea5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:03.604474 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-55599cf3-f79f-49ee-928a-ca8c555bd979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f11853e48a7a4229841ee9c711c7bea5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:03.617347 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d9335ef6-2bcf-42f8-9f65-e0858b352185 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e829698141514a009e2e769854fe66f0', 'request_id': 'req-d9335ef6-2bcf-42f8-9f65-e0858b352185'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:03.619361 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1207/2450] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:03 2022] DELETE /identity/v3/users/f11853e48a7a4229841ee9c711c7bea5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:03.620266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55599cf3-f79f-49ee-928a-ca8c555bd979 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:03.621598 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-55599cf3-f79f-49ee-928a-ca8c555bd979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:03.626593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d9335ef6-2bcf-42f8-9f65-e0858b352185 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e829698141514a009e2e769854fe66f0', 'request_id': 'req-d9335ef6-2bcf-42f8-9f65-e0858b352185'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:03.635369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d9335ef6-2bcf-42f8-9f65-e0858b352185 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e829698141514a009e2e769854fe66f0', 'request_id': 'req-d9335ef6-2bcf-42f8-9f65-e0858b352185'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:03.652192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d9335ef6-2bcf-42f8-9f65-e0858b352185 admin admin] The token cache is being invalidate because project e829698141514a009e2e769854fe66f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:03.652469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d9335ef6-2bcf-42f8-9f65-e0858b352185 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:03.653219 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d9335ef6-2bcf-42f8-9f65-e0858b352185 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:03.654775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1244/2451] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:03 2022] DELETE /identity/v3/projects/e829698141514a009e2e769854fe66f0 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:03.655942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438d66a-6c55-469c-8e62-46e4c3eddb5d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:03.656298 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438d66a-6c55-469c-8e62-46e4c3eddb5d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:03.656568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438d66a-6c55-469c-8e62-46e4c3eddb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:03.678700 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8438d66a-6c55-469c-8e62-46e4c3eddb5d None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:03.679921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8438d66a-6c55-469c-8e62-46e4c3eddb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:03.701116 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55599cf3-f79f-49ee-928a-ca8c555bd979 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:03.706816 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc4382-2274-4f22-a216-41d5b5eb74d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:03.707953 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc4382-2274-4f22-a216-41d5b5eb74d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:03.708328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc4382-2274-4f22-a216-41d5b5eb74d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:03.708718 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc4382-2274-4f22-a216-41d5b5eb74d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:03.709395 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc4382-2274-4f22-a216-41d5b5eb74d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:09:03.709702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc4382-2274-4f22-a216-41d5b5eb74d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:03.711590 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc4382-2274-4f22-a216-41d5b5eb74d6 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:03.724437 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1208/2452] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:09:03.726215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249621da-dfbd-446f-8aac-cad706058d82 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:03.726513 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249621da-dfbd-446f-8aac-cad706058d82 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:03.726790 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249621da-dfbd-446f-8aac-cad706058d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:03.744169 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1245/2453] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:03.745101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438d66a-6c55-469c-8e62-46e4c3eddb5d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:03.746589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8438d66a-6c55-469c-8e62-46e4c3eddb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:03.749002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-249621da-dfbd-446f-8aac-cad706058d82 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:03.750252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-249621da-dfbd-446f-8aac-cad706058d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:03.759557 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438d66a-6c55-469c-8e62-46e4c3eddb5d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:03.760424 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1209/2454] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:03.761258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249621da-dfbd-446f-8aac-cad706058d82 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:03.762605 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-249621da-dfbd-446f-8aac-cad706058d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:03.765321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31fdc1-252c-4e27-a999-9211e27d852c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:03.765968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31fdc1-252c-4e27-a999-9211e27d852c admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:03.766242 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31fdc1-252c-4e27-a999-9211e27d852c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:03.766514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31fdc1-252c-4e27-a999-9211e27d852c admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:03.767180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a31fdc1-252c-4e27-a999-9211e27d852c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:03.769113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a31fdc1-252c-4e27-a999-9211e27d852c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:03.782797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:03.803901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1246/2455] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:03 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:03.805632 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6570c-9ded-4530-bd13-0e43f82ac6ea None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:03.805984 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6570c-9ded-4530-bd13-0e43f82ac6ea None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:03.806282 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6570c-9ded-4530-bd13-0e43f82ac6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:03.812361 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:03.827707 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c3e6570c-9ded-4530-bd13-0e43f82ac6ea None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:03.829106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c3e6570c-9ded-4530-bd13-0e43f82ac6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:03.845564 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249621da-dfbd-446f-8aac-cad706058d82 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:03.904152 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1247/2456] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:03.905083 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6570c-9ded-4530-bd13-0e43f82ac6ea None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:03.911200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe186e7-c5b0-4c4d-ac7a-e55b9ef3ad62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:03.911922 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe186e7-c5b0-4c4d-ac7a-e55b9ef3ad62 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:03.912203 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe186e7-c5b0-4c4d-ac7a-e55b9ef3ad62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:03.912484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe186e7-c5b0-4c4d-ac7a-e55b9ef3ad62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:03.915464 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6570c-9ded-4530-bd13-0e43f82ac6ea None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:03.920313 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7d39e-14a6-4f40-a28e-de57d57ac91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:03.921134 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7d39e-14a6-4f40-a28e-de57d57ac91d admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:03.921264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7d39e-14a6-4f40-a28e-de57d57ac91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:03.921477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7d39e-14a6-4f40-a28e-de57d57ac91d admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:03.922209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7d39e-14a6-4f40-a28e-de57d57ac91d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:03.923108 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe186e7-c5b0-4c4d-ac7a-e55b9ef3ad62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:03.924217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7d39e-14a6-4f40-a28e-de57d57ac91d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:03.924576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe186e7-c5b0-4c4d-ac7a-e55b9ef3ad62 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:03.938948 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1210/2457] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:03.940692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe186e7-c5b0-4c4d-ac7a-e55b9ef3ad62 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:03.952785 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe186e7-c5b0-4c4d-ac7a-e55b9ef3ad62 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:03.953683 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1248/2458] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:03 2022] POST /identity/v3/users => generated 287 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:03.959079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7d39e-14a6-4f40-a28e-de57d57ac91d admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:03.961263 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a1920-308b-42c8-9f6c-ebe5b19dc3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:03.962732 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a1920-308b-42c8-9f6c-ebe5b19dc3bd admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:03.963464 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a1920-308b-42c8-9f6c-ebe5b19dc3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:03.964145 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a1920-308b-42c8-9f6c-ebe5b19dc3bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:03.965660 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a1920-308b-42c8-9f6c-ebe5b19dc3bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:09:03.966424 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a1920-308b-42c8-9f6c-ebe5b19dc3bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:03.968258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a1920-308b-42c8-9f6c-ebe5b19dc3bd admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:03.974110 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7d39e-14a6-4f40-a28e-de57d57ac91d admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:03.980786 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23131e1a-3c4d-4748-abce-d1e43a7c0891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:03.981489 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23131e1a-3c4d-4748-abce-d1e43a7c0891 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:03.981777 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23131e1a-3c4d-4748-abce-d1e43a7c0891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:03.982088 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23131e1a-3c4d-4748-abce-d1e43a7c0891 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:03.982813 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23131e1a-3c4d-4748-abce-d1e43a7c0891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:03.983152 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1211/2459] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:09:03.984561 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23131e1a-3c4d-4748-abce-d1e43a7c0891 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:03.988986 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a1920-308b-42c8-9f6c-ebe5b19dc3bd admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:03.999973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1249/2460] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:09:04.000486 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a1920-308b-42c8-9f6c-ebe5b19dc3bd admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:04.006437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23131e1a-3c4d-4748-abce-d1e43a7c0891 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:04.007074 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62acd1-a8a3-4d51-a9c4-703eac6501d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:04.008071 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62acd1-a8a3-4d51-a9c4-703eac6501d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:04.008749 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62acd1-a8a3-4d51-a9c4-703eac6501d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:04.009010 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62acd1-a8a3-4d51-a9c4-703eac6501d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:04.010196 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62acd1-a8a3-4d51-a9c4-703eac6501d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:04.012979 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62acd1-a8a3-4d51-a9c4-703eac6501d9 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:04.021812 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23131e1a-3c4d-4748-abce-d1e43a7c0891 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:04.030853 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277042fa-579a-4708-8546-f603dc3bc90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:04.031230 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277042fa-579a-4708-8546-f603dc3bc90f admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:04.031561 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277042fa-579a-4708-8546-f603dc3bc90f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:04.031868 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277042fa-579a-4708-8546-f603dc3bc90f admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:04.033371 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277042fa-579a-4708-8546-f603dc3bc90f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:04.035257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277042fa-579a-4708-8546-f603dc3bc90f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:04.048833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1250/2461] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:04.052539 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1212/2462] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:03 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:09:04.053420 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277042fa-579a-4708-8546-f603dc3bc90f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:04.062211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62acd1-a8a3-4d51-a9c4-703eac6501d9 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:04.068834 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277042fa-579a-4708-8546-f603dc3bc90f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:04.155945 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1ea60-1ad3-4e4a-b4fc-957fa51fae2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:04.157456 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1ea60-1ad3-4e4a-b4fc-957fa51fae2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:04.157627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62acd1-a8a3-4d51-a9c4-703eac6501d9 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:04.157858 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1ea60-1ad3-4e4a-b4fc-957fa51fae2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:04.158422 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1ea60-1ad3-4e4a-b4fc-957fa51fae2b admin admin] PATH_INFO: `/v3/projects/02cfd1500ab74c5bb5baadb551a5ef70/users/9f0f5efe137445d48822aca0b3d13c6c/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:04.163567 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec36e8-434d-42cc-8ddb-7a6f97874b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:04.163567 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec36e8-434d-42cc-8ddb-7a6f97874b0f admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:04.163567 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec36e8-434d-42cc-8ddb-7a6f97874b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:04.163986 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec36e8-434d-42cc-8ddb-7a6f97874b0f admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:04.164470 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec36e8-434d-42cc-8ddb-7a6f97874b0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:04.166451 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec36e8-434d-42cc-8ddb-7a6f97874b0f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:04.181335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1ea60-1ad3-4e4a-b4fc-957fa51fae2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=02cfd1500ab74c5bb5baadb551a5ef70, user_id=9f0f5efe137445d48822aca0b3d13c6c, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:04.184545 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1ea60-1ad3-4e4a-b4fc-957fa51fae2b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:04.191182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1213/2463] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:04 2022] POST /identity/v3/users => generated 286 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:04.195505 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec36e8-434d-42cc-8ddb-7a6f97874b0f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:04.203259 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1251/2464] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:04 2022] PUT /identity/v3/projects/02cfd1500ab74c5bb5baadb551a5ef70/users/9f0f5efe137445d48822aca0b3d13c6c/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:04.211374 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec36e8-434d-42cc-8ddb-7a6f97874b0f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:04.264220 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:04.277262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c2a66-5e4f-4d40-bd7f-8c6c89a586ec None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:04.277542 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c2a66-5e4f-4d40-bd7f-8c6c89a586ec None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:04.277814 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c2a66-5e4f-4d40-bd7f-8c6c89a586ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:04.293530 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab290bc-c93a-4c51-9d03-a025792f48ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:04.294184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab290bc-c93a-4c51-9d03-a025792f48ef admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:04.294516 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab290bc-c93a-4c51-9d03-a025792f48ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:04.295056 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab290bc-c93a-4c51-9d03-a025792f48ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:04.296021 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab290bc-c93a-4c51-9d03-a025792f48ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:04.298020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab290bc-c93a-4c51-9d03-a025792f48ef admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:04.311204 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1214/2465] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:09:04.315614 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab290bc-c93a-4c51-9d03-a025792f48ef admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:04.317430 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4ab290bc-c93a-4c51-9d03-a025792f48ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:04.319928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-099c2a66-5e4f-4d40-bd7f-8c6c89a586ec None None] MFA Rules not processed for user `9f0f5efe137445d48822aca0b3d13c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:04.321652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-099c2a66-5e4f-4d40-bd7f-8c6c89a586ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:04.333784 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab290bc-c93a-4c51-9d03-a025792f48ef admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:04.339874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8aa7c4-495f-4e63-8cb8-b17d1e0537af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:04.340578 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8aa7c4-495f-4e63-8cb8-b17d1e0537af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:04.340825 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8aa7c4-495f-4e63-8cb8-b17d1e0537af admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:04.341106 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8aa7c4-495f-4e63-8cb8-b17d1e0537af admin admin] PATH_INFO: `/v3/projects/0e4796a5773e4db48511af809d52da6c` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:04.343368 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8aa7c4-495f-4e63-8cb8-b17d1e0537af admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e4796a5773e4db48511af809d52da6c)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:04.345480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8aa7c4-495f-4e63-8cb8-b17d1e0537af admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:04.352400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:04.411849 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5b8aa7c4-495f-4e63-8cb8-b17d1e0537af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e4796a5773e4db48511af809d52da6c', 'request_id': 'req-5b8aa7c4-495f-4e63-8cb8-b17d1e0537af'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:04.414281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1252/2466] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:04.414281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c2a66-5e4f-4d40-bd7f-8c6c89a586ec None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:04.421233 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5b8aa7c4-495f-4e63-8cb8-b17d1e0537af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e4796a5773e4db48511af809d52da6c', 'request_id': 'req-5b8aa7c4-495f-4e63-8cb8-b17d1e0537af'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:04.428448 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c2a66-5e4f-4d40-bd7f-8c6c89a586ec None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:04.431840 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5b8aa7c4-495f-4e63-8cb8-b17d1e0537af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e4796a5773e4db48511af809d52da6c', 'request_id': 'req-5b8aa7c4-495f-4e63-8cb8-b17d1e0537af'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:04.448030 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5b8aa7c4-495f-4e63-8cb8-b17d1e0537af admin admin] The token cache is being invalidate because project 0e4796a5773e4db48511af809d52da6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:04.448418 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5b8aa7c4-495f-4e63-8cb8-b17d1e0537af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:04.449131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5b8aa7c4-495f-4e63-8cb8-b17d1e0537af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:04.450909 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1215/2467] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:04 2022] DELETE /identity/v3/projects/0e4796a5773e4db48511af809d52da6c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:04.454137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8aa7c4-495f-4e63-8cb8-b17d1e0537af admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:04.455076 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5b8aa7c4-495f-4e63-8cb8-b17d1e0537af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:04.496521 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c7ed1-4fa0-4e01-bdbd-fd452404ace5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:04.497281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c7ed1-4fa0-4e01-bdbd-fd452404ace5 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:04.497569 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c7ed1-4fa0-4e01-bdbd-fd452404ace5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:04.497841 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c7ed1-4fa0-4e01-bdbd-fd452404ace5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:04.498578 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c7ed1-4fa0-4e01-bdbd-fd452404ace5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:04.500195 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c7ed1-4fa0-4e01-bdbd-fd452404ace5 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:04.510899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1253/2468] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:04.514923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c7ed1-4fa0-4e01-bdbd-fd452404ace5 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:04.516687 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e64c7ed1-4fa0-4e01-bdbd-fd452404ace5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:04.526627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8aa7c4-495f-4e63-8cb8-b17d1e0537af admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:04.531579 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba536b7-9d98-4b61-812c-93f94b8e698e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:04.532223 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba536b7-9d98-4b61-812c-93f94b8e698e admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:04.532515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba536b7-9d98-4b61-812c-93f94b8e698e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:04.532975 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba536b7-9d98-4b61-812c-93f94b8e698e admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:04.534284 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba536b7-9d98-4b61-812c-93f94b8e698e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:04.536000 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c7ed1-4fa0-4e01-bdbd-fd452404ace5 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:04.536582 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba536b7-9d98-4b61-812c-93f94b8e698e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:04.543937 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b3026-2b04-456a-b1fd-62e07b4aad12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:04.545039 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b3026-2b04-456a-b1fd-62e07b4aad12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:04.545537 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b3026-2b04-456a-b1fd-62e07b4aad12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:04.546059 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b3026-2b04-456a-b1fd-62e07b4aad12 admin admin] PATH_INFO: `/v3/users/4ebc4a4b21dd4c6cb847c1ad751b105b` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:04.547709 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603b3026-2b04-456a-b1fd-62e07b4aad12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ebc4a4b21dd4c6cb847c1ad751b105b)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:04.550269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603b3026-2b04-456a-b1fd-62e07b4aad12 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:04.569911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1216/2469] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:04 2022] POST /identity/v3/projects => generated 371 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:04.570824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba536b7-9d98-4b61-812c-93f94b8e698e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:04.572499 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2ba536b7-9d98-4b61-812c-93f94b8e698e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:04.602118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-603b3026-2b04-456a-b1fd-62e07b4aad12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ebc4a4b21dd4c6cb847c1ad751b105b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:04.602841 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-603b3026-2b04-456a-b1fd-62e07b4aad12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ebc4a4b21dd4c6cb847c1ad751b105b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:04.611189 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-603b3026-2b04-456a-b1fd-62e07b4aad12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ebc4a4b21dd4c6cb847c1ad751b105b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:04.623795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:04.625124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-603b3026-2b04-456a-b1fd-62e07b4aad12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ebc4a4b21dd4c6cb847c1ad751b105b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:04.633469 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-603b3026-2b04-456a-b1fd-62e07b4aad12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ebc4a4b21dd4c6cb847c1ad751b105b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:04.640500 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1254/2470] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:04 2022] DELETE /identity/v3/users/4ebc4a4b21dd4c6cb847c1ad751b105b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:04.641385 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b3026-2b04-456a-b1fd-62e07b4aad12 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:04.643319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-603b3026-2b04-456a-b1fd-62e07b4aad12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:04.647717 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba536b7-9d98-4b61-812c-93f94b8e698e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:04.694924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:04.728571 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3160c8-6b4a-47f1-a040-6e6710423d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:04.729869 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3160c8-6b4a-47f1-a040-6e6710423d98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:04.730407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3160c8-6b4a-47f1-a040-6e6710423d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:04.730990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3160c8-6b4a-47f1-a040-6e6710423d98 admin admin] PATH_INFO: `/v3/projects/50802b0f362042ef982dcfb1ef5c3744/users/fbba690216b249e6a0791e8e1bba6983/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:04.755013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3160c8-6b4a-47f1-a040-6e6710423d98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50802b0f362042ef982dcfb1ef5c3744, user_id=fbba690216b249e6a0791e8e1bba6983, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:04.758812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3160c8-6b4a-47f1-a040-6e6710423d98 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:04.758995 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b3026-2b04-456a-b1fd-62e07b4aad12 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:04.765912 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bea2dd-123a-4e5d-9371-9e83c483f728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:04.766782 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bea2dd-123a-4e5d-9371-9e83c483f728 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:04.767116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bea2dd-123a-4e5d-9371-9e83c483f728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:04.768246 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bea2dd-123a-4e5d-9371-9e83c483f728 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:04.768932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bea2dd-123a-4e5d-9371-9e83c483f728 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:04.773017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bea2dd-123a-4e5d-9371-9e83c483f728 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:04.781975 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1217/2471] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:04 2022] PUT /identity/v3/projects/50802b0f362042ef982dcfb1ef5c3744/users/fbba690216b249e6a0791e8e1bba6983/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:04.794183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b906eb7-2c1c-4b10-984f-cf894057b71d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:04.796750 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b906eb7-2c1c-4b10-984f-cf894057b71d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:04.796750 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b906eb7-2c1c-4b10-984f-cf894057b71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:04.801087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1255/2472] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:04 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:04.805744 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bea2dd-123a-4e5d-9371-9e83c483f728 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:04.821486 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bea2dd-123a-4e5d-9371-9e83c483f728 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:04.866076 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-9b906eb7-2c1c-4b10-984f-cf894057b71d None None] MFA Rules not processed for user `fbba690216b249e6a0791e8e1bba6983`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:04.868392 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9b906eb7-2c1c-4b10-984f-cf894057b71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:04.934859 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce05050-e961-4017-9295-8e6af04218f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:04.935846 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce05050-e961-4017-9295-8e6af04218f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:04.936318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce05050-e961-4017-9295-8e6af04218f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:04.936761 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce05050-e961-4017-9295-8e6af04218f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:04.938146 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce05050-e961-4017-9295-8e6af04218f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:04.940919 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce05050-e961-4017-9295-8e6af04218f1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:04.961287 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1256/2473] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:04.966838 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce05050-e961-4017-9295-8e6af04218f1 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:04.989378 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1218/2474] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:09:04.989932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce05050-e961-4017-9295-8e6af04218f1 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:04.996976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1846e7-38cc-446c-bea5-721c249a7667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:04.997977 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1846e7-38cc-446c-bea5-721c249a7667 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:04.998370 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1846e7-38cc-446c-bea5-721c249a7667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:04.998754 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1846e7-38cc-446c-bea5-721c249a7667 admin admin] PATH_INFO: `/v3/projects/8b2c9c9893c8405c9ede18dc727163fe/users/04988c350e6f44f6ad4b81c9b5a2d42d/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:05.011796 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b906eb7-2c1c-4b10-984f-cf894057b71d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:05.014957 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9b906eb7-2c1c-4b10-984f-cf894057b71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:05.015741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1846e7-38cc-446c-bea5-721c249a7667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b2c9c9893c8405c9ede18dc727163fe, user_id=04988c350e6f44f6ad4b81c9b5a2d42d, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:05.019151 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1846e7-38cc-446c-bea5-721c249a7667 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:05.042215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1257/2475] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:04 2022] PUT /identity/v3/projects/8b2c9c9893c8405c9ede18dc727163fe/users/04988c350e6f44f6ad4b81c9b5a2d42d/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:05.047413 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1846e7-38cc-446c-bea5-721c249a7667 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:05.122419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:05.134745 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b906eb7-2c1c-4b10-984f-cf894057b71d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:05.152052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1846e7-38cc-446c-bea5-721c249a7667 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:05.156876 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a7dbe-1697-467c-b8db-5dc84b6df692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:05.157506 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a7dbe-1697-467c-b8db-5dc84b6df692 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:05.157788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a7dbe-1697-467c-b8db-5dc84b6df692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:05.158094 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a7dbe-1697-467c-b8db-5dc84b6df692 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:05.158833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a7dbe-1697-467c-b8db-5dc84b6df692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:05.160455 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a7dbe-1697-467c-b8db-5dc84b6df692 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:05.176194 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1258/2476] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:05.176194 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a7dbe-1697-467c-b8db-5dc84b6df692 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:05.176540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fb1a7dbe-1697-467c-b8db-5dc84b6df692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:05.194282 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a7dbe-1697-467c-b8db-5dc84b6df692 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:05.201921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d811d-9c93-4188-8a6b-98fcd84220c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:05.202929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d811d-9c93-4188-8a6b-98fcd84220c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:05.203390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d811d-9c93-4188-8a6b-98fcd84220c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:05.203845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d811d-9c93-4188-8a6b-98fcd84220c1 admin admin] PATH_INFO: `/v3/projects/82e81dc435d94497b2559b2b41d31baa` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:05.206411 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d811d-9c93-4188-8a6b-98fcd84220c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82e81dc435d94497b2559b2b41d31baa)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:05.209695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d811d-9c93-4188-8a6b-98fcd84220c1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:05.231843 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71847c25-cef2-4e58-b91d-209fc95ba389 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:05.232597 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71847c25-cef2-4e58-b91d-209fc95ba389 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:05.232993 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71847c25-cef2-4e58-b91d-209fc95ba389 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:05.233374 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71847c25-cef2-4e58-b91d-209fc95ba389 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:05.245222 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:05.266539 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a84d811d-9c93-4188-8a6b-98fcd84220c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82e81dc435d94497b2559b2b41d31baa', 'request_id': 'req-a84d811d-9c93-4188-8a6b-98fcd84220c1'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:05.273666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a84d811d-9c93-4188-8a6b-98fcd84220c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82e81dc435d94497b2559b2b41d31baa', 'request_id': 'req-a84d811d-9c93-4188-8a6b-98fcd84220c1'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:05.281164 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a84d811d-9c93-4188-8a6b-98fcd84220c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82e81dc435d94497b2559b2b41d31baa', 'request_id': 'req-a84d811d-9c93-4188-8a6b-98fcd84220c1'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:05.296917 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a84d811d-9c93-4188-8a6b-98fcd84220c1 admin admin] The token cache is being invalidate because project 82e81dc435d94497b2559b2b41d31baa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:05.296917 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a84d811d-9c93-4188-8a6b-98fcd84220c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:05.296917 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a84d811d-9c93-4188-8a6b-98fcd84220c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:05.298755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1259/2477] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:05 2022] DELETE /identity/v3/projects/82e81dc435d94497b2559b2b41d31baa => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:05.299639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d811d-9c93-4188-8a6b-98fcd84220c1 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:05.301262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a84d811d-9c93-4188-8a6b-98fcd84220c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:05.306323 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71847c25-cef2-4e58-b91d-209fc95ba389 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:05.307977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71847c25-cef2-4e58-b91d-209fc95ba389 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:05.318334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1219/2478] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:09:05.319849 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361138ea-d59e-4b57-8f7f-0ce7ccc68069 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:05.320543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361138ea-d59e-4b57-8f7f-0ce7ccc68069 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:05.320543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361138ea-d59e-4b57-8f7f-0ce7ccc68069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:05.343150 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-361138ea-d59e-4b57-8f7f-0ce7ccc68069 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:05.344605 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-361138ea-d59e-4b57-8f7f-0ce7ccc68069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:05.377714 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d811d-9c93-4188-8a6b-98fcd84220c1 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:05.382872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3a5e4-167d-42f4-ae6b-23de4eeabb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:05.383731 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3a5e4-167d-42f4-ae6b-23de4eeabb0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:05.384020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3a5e4-167d-42f4-ae6b-23de4eeabb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:05.384299 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3a5e4-167d-42f4-ae6b-23de4eeabb0f admin admin] PATH_INFO: `/v3/projects/8b2c9c9893c8405c9ede18dc727163fe/users/04988c350e6f44f6ad4b81c9b5a2d42d/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:05.386390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3a5e4-167d-42f4-ae6b-23de4eeabb0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b2c9c9893c8405c9ede18dc727163fe, user_id=04988c350e6f44f6ad4b81c9b5a2d42d, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:05.389135 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3a5e4-167d-42f4-ae6b-23de4eeabb0f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:05.401973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1260/2479] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:05 2022] PUT /identity/v3/projects/8b2c9c9893c8405c9ede18dc727163fe/users/04988c350e6f44f6ad4b81c9b5a2d42d/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:05.412848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91687b-a6a7-40b9-a254-31ee770a885b None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:05.414408 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91687b-a6a7-40b9-a254-31ee770a885b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:05.414916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91687b-a6a7-40b9-a254-31ee770a885b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:05.416446 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1220/2480] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:05.426183 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522f7e1-369b-448d-85e6-790483625309 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:05.426477 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522f7e1-369b-448d-85e6-790483625309 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:05.426752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522f7e1-369b-448d-85e6-790483625309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:05.448066 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-9522f7e1-369b-448d-85e6-790483625309 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:05.449285 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9522f7e1-369b-448d-85e6-790483625309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:05.450265 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-aa91687b-a6a7-40b9-a254-31ee770a885b None None] MFA Rules not processed for user `04988c350e6f44f6ad4b81c9b5a2d42d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:05.451474 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-aa91687b-a6a7-40b9-a254-31ee770a885b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:05.493434 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:05.531167 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1221/2481] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:09:05.535999 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522f7e1-369b-448d-85e6-790483625309 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:05.546394 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522f7e1-369b-448d-85e6-790483625309 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:05.551966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5884e17-e723-46b0-bea2-0684dce539e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:05.552750 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5884e17-e723-46b0-bea2-0684dce539e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:05.553368 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5884e17-e723-46b0-bea2-0684dce539e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:05.553651 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5884e17-e723-46b0-bea2-0684dce539e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:05.554406 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5884e17-e723-46b0-bea2-0684dce539e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:09:05.554713 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5884e17-e723-46b0-bea2-0684dce539e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:05.556502 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5884e17-e723-46b0-bea2-0684dce539e4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:05.559247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1261/2482] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:05 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:09:05.571838 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1222/2483] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:09:05.573518 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91687b-a6a7-40b9-a254-31ee770a885b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:05.575134 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-aa91687b-a6a7-40b9-a254-31ee770a885b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:05.576037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5884e17-e723-46b0-bea2-0684dce539e4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:05.587627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5884e17-e723-46b0-bea2-0684dce539e4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:05.593387 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf5719-629f-41a3-a64d-80ccec4368f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:05.594755 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf5719-629f-41a3-a64d-80ccec4368f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:05.594755 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf5719-629f-41a3-a64d-80ccec4368f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:05.595448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf5719-629f-41a3-a64d-80ccec4368f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:05.595946 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf5719-629f-41a3-a64d-80ccec4368f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:05.598478 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf5719-629f-41a3-a64d-80ccec4368f1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:05.621363 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:05.645262 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1223/2484] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:05 2022] POST /identity/v3/projects => generated 371 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:09:05.650234 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf5719-629f-41a3-a64d-80ccec4368f1 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:05.661796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91687b-a6a7-40b9-a254-31ee770a885b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:05.749071 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf5719-629f-41a3-a64d-80ccec4368f1 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:05.756764 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca435615-1061-4e1e-8eac-bc8a4cf48445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:05.757625 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca435615-1061-4e1e-8eac-bc8a4cf48445 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:05.757934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca435615-1061-4e1e-8eac-bc8a4cf48445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:05.758226 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca435615-1061-4e1e-8eac-bc8a4cf48445 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:05.758945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca435615-1061-4e1e-8eac-bc8a4cf48445 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:05.759721 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec8e55-711f-4a92-bdde-0c2e4225f0d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:05.760584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec8e55-711f-4a92-bdde-0c2e4225f0d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:05.760960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec8e55-711f-4a92-bdde-0c2e4225f0d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:05.761316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec8e55-711f-4a92-bdde-0c2e4225f0d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:05.764681 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca435615-1061-4e1e-8eac-bc8a4cf48445 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:05.793561 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1224/2485] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:05 2022] POST /identity/v3/users => generated 286 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:05.797673 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca435615-1061-4e1e-8eac-bc8a4cf48445 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:05.808289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca435615-1061-4e1e-8eac-bc8a4cf48445 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:05.813098 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c1b8a-81fc-4e1b-9446-7276772e2566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:05.813721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c1b8a-81fc-4e1b-9446-7276772e2566 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:05.814048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c1b8a-81fc-4e1b-9446-7276772e2566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:05.814319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c1b8a-81fc-4e1b-9446-7276772e2566 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:05.815104 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c1b8a-81fc-4e1b-9446-7276772e2566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:05.816774 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c1b8a-81fc-4e1b-9446-7276772e2566 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:05.831206 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1225/2486] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:05.832298 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c1b8a-81fc-4e1b-9446-7276772e2566 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:05.843647 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c1b8a-81fc-4e1b-9446-7276772e2566 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:05.847202 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec8e55-711f-4a92-bdde-0c2e4225f0d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:05.849367 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec8e55-711f-4a92-bdde-0c2e4225f0d4 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:05.850527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ce99a-be3a-4393-aceb-b79d21962c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:05.851151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ce99a-be3a-4393-aceb-b79d21962c26 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:05.851432 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ce99a-be3a-4393-aceb-b79d21962c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:05.851707 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ce99a-be3a-4393-aceb-b79d21962c26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:05.852403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ce99a-be3a-4393-aceb-b79d21962c26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:05.854035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ce99a-be3a-4393-aceb-b79d21962c26 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:05.861376 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1262/2487] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:05 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:05.873926 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1226/2488] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:05.875644 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec8e55-711f-4a92-bdde-0c2e4225f0d4 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:05.888797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec8e55-711f-4a92-bdde-0c2e4225f0d4 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:05.894174 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3dd607-fed4-4921-97f2-85113cebacfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:05.894858 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dd607-fed4-4921-97f2-85113cebacfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:05.895153 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dd607-fed4-4921-97f2-85113cebacfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:05.895478 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dd607-fed4-4921-97f2-85113cebacfb admin admin] PATH_INFO: `/v3/projects/61f2cd109b8f430e85330b2d5f5eab92/users/137b8d6cd7424a5fa3431466aa518ab4/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:05.913989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3dd607-fed4-4921-97f2-85113cebacfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=61f2cd109b8f430e85330b2d5f5eab92, user_id=137b8d6cd7424a5fa3431466aa518ab4, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:05.916656 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3dd607-fed4-4921-97f2-85113cebacfb admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:05.931897 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1263/2489] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:05 2022] PUT /identity/v3/projects/61f2cd109b8f430e85330b2d5f5eab92/users/137b8d6cd7424a5fa3431466aa518ab4/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:05.943687 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a5b1e-ed2a-4fa2-885a-67f005137e11 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:05.943972 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a5b1e-ed2a-4fa2-885a-67f005137e11 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:05.944243 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a5b1e-ed2a-4fa2-885a-67f005137e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:05.977586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-489a5b1e-ed2a-4fa2-885a-67f005137e11 None None] MFA Rules not processed for user `137b8d6cd7424a5fa3431466aa518ab4`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:05.979887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-489a5b1e-ed2a-4fa2-885a-67f005137e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:06.077096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1227/2490] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:06.091914 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3dd607-fed4-4921-97f2-85113cebacfb admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:06.165195 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3dd607-fed4-4921-97f2-85113cebacfb admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:06.170741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c3c7d-b8c1-4d54-8226-b8a24efbdcb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:06.171582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c3c7d-b8c1-4d54-8226-b8a24efbdcb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:06.171729 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c3c7d-b8c1-4d54-8226-b8a24efbdcb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:06.172017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c3c7d-b8c1-4d54-8226-b8a24efbdcb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:06.182134 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c3c7d-b8c1-4d54-8226-b8a24efbdcb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:06.183695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c3c7d-b8c1-4d54-8226-b8a24efbdcb0 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:06.195630 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1264/2491] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:09:06.222229 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a5b1e-ed2a-4fa2-885a-67f005137e11 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:06.227317 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:06.313932 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a5b1e-ed2a-4fa2-885a-67f005137e11 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:06.322080 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811ad33-8229-4ed2-abf8-49453cc0a9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:06.323113 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811ad33-8229-4ed2-abf8-49453cc0a9dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:06.323578 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811ad33-8229-4ed2-abf8-49453cc0a9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:06.324011 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811ad33-8229-4ed2-abf8-49453cc0a9dd admin admin] PATH_INFO: `/v3/users/137b8d6cd7424a5fa3431466aa518ab4` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:06.326130 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811ad33-8229-4ed2-abf8-49453cc0a9dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=137b8d6cd7424a5fa3431466aa518ab4)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:06.329482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811ad33-8229-4ed2-abf8-49453cc0a9dd admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:06.350820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c3c7d-b8c1-4d54-8226-b8a24efbdcb0 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:06.367484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c3c7d-b8c1-4d54-8226-b8a24efbdcb0 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:06.373631 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75c238-9250-4187-9f9f-00e172795178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:06.374650 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75c238-9250-4187-9f9f-00e172795178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:06.375028 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75c238-9250-4187-9f9f-00e172795178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:06.375402 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75c238-9250-4187-9f9f-00e172795178 admin admin] PATH_INFO: `/v3/users/9f0f5efe137445d48822aca0b3d13c6c` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:06.376827 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75c238-9250-4187-9f9f-00e172795178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f0f5efe137445d48822aca0b3d13c6c)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:06.379792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75c238-9250-4187-9f9f-00e172795178 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:06.396575 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:06.401093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c811ad33-8229-4ed2-abf8-49453cc0a9dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '137b8d6cd7424a5fa3431466aa518ab4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:06.402001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c811ad33-8229-4ed2-abf8-49453cc0a9dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '137b8d6cd7424a5fa3431466aa518ab4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:06.413496 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c811ad33-8229-4ed2-abf8-49453cc0a9dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '137b8d6cd7424a5fa3431466aa518ab4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:06.430689 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c811ad33-8229-4ed2-abf8-49453cc0a9dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '137b8d6cd7424a5fa3431466aa518ab4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:06.431670 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c811ad33-8229-4ed2-abf8-49453cc0a9dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '137b8d6cd7424a5fa3431466aa518ab4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:06.440833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0a75c238-9250-4187-9f9f-00e172795178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f0f5efe137445d48822aca0b3d13c6c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:06.441308 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0a75c238-9250-4187-9f9f-00e172795178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f0f5efe137445d48822aca0b3d13c6c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:06.448512 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1228/2492] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:06 2022] DELETE /identity/v3/users/137b8d6cd7424a5fa3431466aa518ab4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:06.452395 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0a75c238-9250-4187-9f9f-00e172795178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f0f5efe137445d48822aca0b3d13c6c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:06.454058 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973b8d9-1cbc-451f-b689-d543c66f8cfe None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:06.454417 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973b8d9-1cbc-451f-b689-d543c66f8cfe None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:06.454775 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973b8d9-1cbc-451f-b689-d543c66f8cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:06.465449 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0a75c238-9250-4187-9f9f-00e172795178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f0f5efe137445d48822aca0b3d13c6c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:06.474427 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0a75c238-9250-4187-9f9f-00e172795178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f0f5efe137445d48822aca0b3d13c6c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:06.478192 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7973b8d9-1cbc-451f-b689-d543c66f8cfe None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:06.479722 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7973b8d9-1cbc-451f-b689-d543c66f8cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:06.482059 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1265/2493] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:06 2022] DELETE /identity/v3/users/9f0f5efe137445d48822aca0b3d13c6c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:06.488270 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd585ac1-ed26-4739-bafa-6b2a58a63896 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:06.488676 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd585ac1-ed26-4739-bafa-6b2a58a63896 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:06.489123 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd585ac1-ed26-4739-bafa-6b2a58a63896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:06.515287 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-bd585ac1-ed26-4739-bafa-6b2a58a63896 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:06.516673 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-bd585ac1-ed26-4739-bafa-6b2a58a63896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:06.557107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1229/2494] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:06.568165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973b8d9-1cbc-451f-b689-d543c66f8cfe None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:06.570029 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7973b8d9-1cbc-451f-b689-d543c66f8cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:06.592385 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1266/2495] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:09:06.600285 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:06.607262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd585ac1-ed26-4739-bafa-6b2a58a63896 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:06.610987 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-bd585ac1-ed26-4739-bafa-6b2a58a63896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:06.641273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973b8d9-1cbc-451f-b689-d543c66f8cfe None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:06.645289 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e2aca-966e-43c9-8ab0-fc01f4e8349f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:06.646033 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e2aca-966e-43c9-8ab0-fc01f4e8349f service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:06.646328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e2aca-966e-43c9-8ab0-fc01f4e8349f service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:06.646601 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e2aca-966e-43c9-8ab0-fc01f4e8349f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:06.657511 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e2aca-966e-43c9-8ab0-fc01f4e8349f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:06.659673 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e2aca-966e-43c9-8ab0-fc01f4e8349f service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:06.671079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1230/2496] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:06.685436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd585ac1-ed26-4739-bafa-6b2a58a63896 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:06.689553 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10610e4a-cdff-486a-8328-dc4bfda960d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:06.690130 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10610e4a-cdff-486a-8328-dc4bfda960d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:06.690408 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10610e4a-cdff-486a-8328-dc4bfda960d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:06.690683 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10610e4a-cdff-486a-8328-dc4bfda960d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:06.702179 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10610e4a-cdff-486a-8328-dc4bfda960d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:06.703981 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10610e4a-cdff-486a-8328-dc4bfda960d4 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:06.714740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1267/2497] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:07.072422 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e2aca-966e-43c9-8ab0-fc01f4e8349f service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:07.074445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2f9e2aca-966e-43c9-8ab0-fc01f4e8349f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:07.091742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e2aca-966e-43c9-8ab0-fc01f4e8349f service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:07.097404 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0b02c-4e40-48ef-ae6c-1817785f8ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:07.098460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0b02c-4e40-48ef-ae6c-1817785f8ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:07.098933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0b02c-4e40-48ef-ae6c-1817785f8ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:07.099218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0b02c-4e40-48ef-ae6c-1817785f8ed7 admin admin] PATH_INFO: `/v3/projects/61f2cd109b8f430e85330b2d5f5eab92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:07.100501 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0b02c-4e40-48ef-ae6c-1817785f8ed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61f2cd109b8f430e85330b2d5f5eab92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:07.102673 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0b02c-4e40-48ef-ae6c-1817785f8ed7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:07.169199 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e6d0b02c-4e40-48ef-ae6c-1817785f8ed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61f2cd109b8f430e85330b2d5f5eab92', 'request_id': 'req-e6d0b02c-4e40-48ef-ae6c-1817785f8ed7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:07.179714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e6d0b02c-4e40-48ef-ae6c-1817785f8ed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61f2cd109b8f430e85330b2d5f5eab92', 'request_id': 'req-e6d0b02c-4e40-48ef-ae6c-1817785f8ed7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:07.191535 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10610e4a-cdff-486a-8328-dc4bfda960d4 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:07.193704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-10610e4a-cdff-486a-8328-dc4bfda960d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:07.194633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e6d0b02c-4e40-48ef-ae6c-1817785f8ed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61f2cd109b8f430e85330b2d5f5eab92', 'request_id': 'req-e6d0b02c-4e40-48ef-ae6c-1817785f8ed7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:07.209171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10610e4a-cdff-486a-8328-dc4bfda960d4 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:07.222404 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e6d0b02c-4e40-48ef-ae6c-1817785f8ed7 admin admin] The token cache is being invalidate because project 61f2cd109b8f430e85330b2d5f5eab92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:07.222972 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e6d0b02c-4e40-48ef-ae6c-1817785f8ed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:07.225062 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e6d0b02c-4e40-48ef-ae6c-1817785f8ed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:07.231774 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1231/2498] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:07 2022] DELETE /identity/v3/projects/61f2cd109b8f430e85330b2d5f5eab92 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:07.236021 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a7bc6-ace1-499e-ab81-a00cc0e0aace None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:07.236495 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a7bc6-ace1-499e-ab81-a00cc0e0aace None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:07.237537 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a7bc6-ace1-499e-ab81-a00cc0e0aace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:07.260942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:07.261553 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-963a7bc6-ace1-499e-ab81-a00cc0e0aace None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:07.262865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-963a7bc6-ace1-499e-ab81-a00cc0e0aace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:07.300804 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d33bd0-9113-4f0b-bb46-5e9ddbc85edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:07.301927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d33bd0-9113-4f0b-bb46-5e9ddbc85edd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:07.302155 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d33bd0-9113-4f0b-bb46-5e9ddbc85edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:07.302403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d33bd0-9113-4f0b-bb46-5e9ddbc85edd admin admin] PATH_INFO: `/v3/projects/02cfd1500ab74c5bb5baadb551a5ef70` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:07.303873 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d33bd0-9113-4f0b-bb46-5e9ddbc85edd admin admin] RBAC: Authorizing `identity:delete_project(project_id=02cfd1500ab74c5bb5baadb551a5ef70)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:07.306126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d33bd0-9113-4f0b-bb46-5e9ddbc85edd admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:07.365207 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1232/2499] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:09:07.377037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-21d33bd0-9113-4f0b-bb46-5e9ddbc85edd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02cfd1500ab74c5bb5baadb551a5ef70', 'request_id': 'req-21d33bd0-9113-4f0b-bb46-5e9ddbc85edd'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:07.387891 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-21d33bd0-9113-4f0b-bb46-5e9ddbc85edd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02cfd1500ab74c5bb5baadb551a5ef70', 'request_id': 'req-21d33bd0-9113-4f0b-bb46-5e9ddbc85edd'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:07.399405 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-21d33bd0-9113-4f0b-bb46-5e9ddbc85edd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02cfd1500ab74c5bb5baadb551a5ef70', 'request_id': 'req-21d33bd0-9113-4f0b-bb46-5e9ddbc85edd'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:07.419617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-21d33bd0-9113-4f0b-bb46-5e9ddbc85edd admin admin] The token cache is being invalidate because project 02cfd1500ab74c5bb5baadb551a5ef70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:07.420256 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-21d33bd0-9113-4f0b-bb46-5e9ddbc85edd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:07.421403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-21d33bd0-9113-4f0b-bb46-5e9ddbc85edd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:07.423281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1268/2500] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:07 2022] DELETE /identity/v3/projects/02cfd1500ab74c5bb5baadb551a5ef70 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:07.438234 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a7bc6-ace1-499e-ab81-a00cc0e0aace None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:07.440394 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-963a7bc6-ace1-499e-ab81-a00cc0e0aace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:07.446611 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa976810-b6b6-49ce-8001-66b04ca78084 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:07.447070 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa976810-b6b6-49ce-8001-66b04ca78084 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:07.447548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa976810-b6b6-49ce-8001-66b04ca78084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:07.454014 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a7bc6-ace1-499e-ab81-a00cc0e0aace None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:07.471811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-fa976810-b6b6-49ce-8001-66b04ca78084 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:07.473651 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fa976810-b6b6-49ce-8001-66b04ca78084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:07.519077 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:07.539257 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f8c48-3911-49b6-a106-3b467e0ffc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:07.540389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f8c48-3911-49b6-a106-3b467e0ffc59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:07.540873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f8c48-3911-49b6-a106-3b467e0ffc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:07.541351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f8c48-3911-49b6-a106-3b467e0ffc59 admin admin] PATH_INFO: `/v3/users/04988c350e6f44f6ad4b81c9b5a2d42d` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:07.543452 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f8c48-3911-49b6-a106-3b467e0ffc59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04988c350e6f44f6ad4b81c9b5a2d42d)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:07.544304 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1269/2501] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:09:07.547383 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f8c48-3911-49b6-a106-3b467e0ffc59 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:07.548610 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa976810-b6b6-49ce-8001-66b04ca78084 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:07.558599 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa976810-b6b6-49ce-8001-66b04ca78084 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:07.563704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55709a47-2e71-4612-a74b-b36adb0e6137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:07.564332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55709a47-2e71-4612-a74b-b36adb0e6137 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:07.564639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55709a47-2e71-4612-a74b-b36adb0e6137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:07.564916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55709a47-2e71-4612-a74b-b36adb0e6137 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:07.565641 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55709a47-2e71-4612-a74b-b36adb0e6137 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:09:07.565959 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55709a47-2e71-4612-a74b-b36adb0e6137 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:07.567577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55709a47-2e71-4612-a74b-b36adb0e6137 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:07.580858 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1270/2502] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:09:07.585859 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55709a47-2e71-4612-a74b-b36adb0e6137 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:07.605304 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55709a47-2e71-4612-a74b-b36adb0e6137 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:07.617516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b7cf4f-aafb-4772-9e53-0119257051df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:07.620066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7cf4f-aafb-4772-9e53-0119257051df admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:07.620066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7cf4f-aafb-4772-9e53-0119257051df admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:07.620066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7cf4f-aafb-4772-9e53-0119257051df admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:07.621386 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b7cf4f-aafb-4772-9e53-0119257051df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:07.623861 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b7cf4f-aafb-4772-9e53-0119257051df admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:07.639008 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2f4f8c48-3911-49b6-a106-3b467e0ffc59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04988c350e6f44f6ad4b81c9b5a2d42d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:07.640121 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2f4f8c48-3911-49b6-a106-3b467e0ffc59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04988c350e6f44f6ad4b81c9b5a2d42d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:07.652985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2f4f8c48-3911-49b6-a106-3b467e0ffc59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04988c350e6f44f6ad4b81c9b5a2d42d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:07.660579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1271/2503] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:07 2022] POST /identity/v3/projects => generated 375 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:09:07.666832 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b7cf4f-aafb-4772-9e53-0119257051df admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:07.672659 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2f4f8c48-3911-49b6-a106-3b467e0ffc59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04988c350e6f44f6ad4b81c9b5a2d42d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:07.673414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2f4f8c48-3911-49b6-a106-3b467e0ffc59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04988c350e6f44f6ad4b81c9b5a2d42d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:07.687586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1233/2504] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:07 2022] DELETE /identity/v3/users/04988c350e6f44f6ad4b81c9b5a2d42d => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:07.721800 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:07.762221 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b7cf4f-aafb-4772-9e53-0119257051df admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:07.840002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d354b-e054-482e-931b-6afa81e45f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:07.840558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d354b-e054-482e-931b-6afa81e45f37 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:07.840850 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d354b-e054-482e-931b-6afa81e45f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:07.841316 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d354b-e054-482e-931b-6afa81e45f37 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:07.842247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d354b-e054-482e-931b-6afa81e45f37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:07.844233 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d354b-e054-482e-931b-6afa81e45f37 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:07.873006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1272/2505] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:07 2022] POST /identity/v3/users => generated 288 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:07.877008 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f8c48-3911-49b6-a106-3b467e0ffc59 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:07.878992 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2f4f8c48-3911-49b6-a106-3b467e0ffc59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:07.900132 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f8c48-3911-49b6-a106-3b467e0ffc59 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:07.905385 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece17038-8d61-4aa7-880e-d81315cc5c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:07.906212 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece17038-8d61-4aa7-880e-d81315cc5c0f admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:07.906479 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece17038-8d61-4aa7-880e-d81315cc5c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:07.906755 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece17038-8d61-4aa7-880e-d81315cc5c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:07.907506 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece17038-8d61-4aa7-880e-d81315cc5c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:07.909741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece17038-8d61-4aa7-880e-d81315cc5c0f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:07.922424 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1234/2506] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:07.926846 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d354b-e054-482e-931b-6afa81e45f37 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:07.937394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d354b-e054-482e-931b-6afa81e45f37 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:07.942602 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780827b2-eb5e-46f1-917e-8aac6f6b2345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:07.943239 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780827b2-eb5e-46f1-917e-8aac6f6b2345 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:07.943523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780827b2-eb5e-46f1-917e-8aac6f6b2345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:07.943801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780827b2-eb5e-46f1-917e-8aac6f6b2345 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:07.944508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780827b2-eb5e-46f1-917e-8aac6f6b2345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:07.946357 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780827b2-eb5e-46f1-917e-8aac6f6b2345 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:07.958474 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1273/2507] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:07.963027 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece17038-8d61-4aa7-880e-d81315cc5c0f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:07.974256 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece17038-8d61-4aa7-880e-d81315cc5c0f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:07.987485 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ddb73d-daf6-458e-b6c3-07e119308293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:07.988190 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddb73d-daf6-458e-b6c3-07e119308293 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:07.988473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddb73d-daf6-458e-b6c3-07e119308293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:07.988750 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddb73d-daf6-458e-b6c3-07e119308293 admin admin] PATH_INFO: `/v3/projects/cb564e3e495b49eea4b0e2477e812577/users/e23c38fff394405c99b924d247d0d70b/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:08.009042 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ddb73d-daf6-458e-b6c3-07e119308293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb564e3e495b49eea4b0e2477e812577, user_id=e23c38fff394405c99b924d247d0d70b, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:08.012045 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ddb73d-daf6-458e-b6c3-07e119308293 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:08.049624 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1235/2508] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:07 2022] PUT /identity/v3/projects/cb564e3e495b49eea4b0e2477e812577/users/e23c38fff394405c99b924d247d0d70b/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:08.050250 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780827b2-eb5e-46f1-917e-8aac6f6b2345 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:08.050250 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-780827b2-eb5e-46f1-917e-8aac6f6b2345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:08.055860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc19887-c4e6-4ad0-91ab-aab9375ac5d6 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:08.056292 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc19887-c4e6-4ad0-91ab-aab9375ac5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:08.056766 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc19887-c4e6-4ad0-91ab-aab9375ac5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:08.064626 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780827b2-eb5e-46f1-917e-8aac6f6b2345 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:08.124164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-9cc19887-c4e6-4ad0-91ab-aab9375ac5d6 None None] MFA Rules not processed for user `e23c38fff394405c99b924d247d0d70b`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:08.125908 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9cc19887-c4e6-4ad0-91ab-aab9375ac5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:08.202647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04625e7e-6dbc-4f0f-b027-805dd6e411f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:08.203722 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04625e7e-6dbc-4f0f-b027-805dd6e411f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:08.204228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04625e7e-6dbc-4f0f-b027-805dd6e411f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:08.204726 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04625e7e-6dbc-4f0f-b027-805dd6e411f0 admin admin] PATH_INFO: `/v3/projects/8b2c9c9893c8405c9ede18dc727163fe` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:08.207419 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04625e7e-6dbc-4f0f-b027-805dd6e411f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b2c9c9893c8405c9ede18dc727163fe)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:08.211501 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04625e7e-6dbc-4f0f-b027-805dd6e411f0 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:08.223852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1236/2509] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:08.224819 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc19887-c4e6-4ad0-91ab-aab9375ac5d6 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:08.226244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9cc19887-c4e6-4ad0-91ab-aab9375ac5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:08.246124 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc19887-c4e6-4ad0-91ab-aab9375ac5d6 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:08.255610 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807cfa29-dcb2-4eb8-92e6-dad6748574c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:08.256293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cfa29-dcb2-4eb8-92e6-dad6748574c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:08.256579 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cfa29-dcb2-4eb8-92e6-dad6748574c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:08.256853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cfa29-dcb2-4eb8-92e6-dad6748574c8 admin admin] PATH_INFO: `/v3/users/fbba690216b249e6a0791e8e1bba6983` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:08.258197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807cfa29-dcb2-4eb8-92e6-dad6748574c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbba690216b249e6a0791e8e1bba6983)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:08.260386 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807cfa29-dcb2-4eb8-92e6-dad6748574c8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:08.279843 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:08.281675 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-04625e7e-6dbc-4f0f-b027-805dd6e411f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b2c9c9893c8405c9ede18dc727163fe', 'request_id': 'req-04625e7e-6dbc-4f0f-b027-805dd6e411f0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:08.290596 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:08.292671 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-04625e7e-6dbc-4f0f-b027-805dd6e411f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b2c9c9893c8405c9ede18dc727163fe', 'request_id': 'req-04625e7e-6dbc-4f0f-b027-805dd6e411f0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:08.303436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-04625e7e-6dbc-4f0f-b027-805dd6e411f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b2c9c9893c8405c9ede18dc727163fe', 'request_id': 'req-04625e7e-6dbc-4f0f-b027-805dd6e411f0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:08.318664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-04625e7e-6dbc-4f0f-b027-805dd6e411f0 admin admin] The token cache is being invalidate because project 8b2c9c9893c8405c9ede18dc727163fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:08.319047 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-04625e7e-6dbc-4f0f-b027-805dd6e411f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:08.319766 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-04625e7e-6dbc-4f0f-b027-805dd6e411f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:08.321434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1274/2510] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:08 2022] DELETE /identity/v3/projects/8b2c9c9893c8405c9ede18dc727163fe => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:08.322234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04625e7e-6dbc-4f0f-b027-805dd6e411f0 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:08.323783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-04625e7e-6dbc-4f0f-b027-805dd6e411f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:08.327260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-807cfa29-dcb2-4eb8-92e6-dad6748574c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbba690216b249e6a0791e8e1bba6983'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:08.328119 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-807cfa29-dcb2-4eb8-92e6-dad6748574c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbba690216b249e6a0791e8e1bba6983'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:08.339018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-807cfa29-dcb2-4eb8-92e6-dad6748574c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbba690216b249e6a0791e8e1bba6983'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:08.354597 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-807cfa29-dcb2-4eb8-92e6-dad6748574c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbba690216b249e6a0791e8e1bba6983'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:08.355407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-807cfa29-dcb2-4eb8-92e6-dad6748574c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbba690216b249e6a0791e8e1bba6983'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:08.369601 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1237/2511] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:08 2022] DELETE /identity/v3/users/fbba690216b249e6a0791e8e1bba6983 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:08.370914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f9202-4072-4959-9836-e2d5a8c5f388 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:08.371197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f9202-4072-4959-9836-e2d5a8c5f388 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:08.371466 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f9202-4072-4959-9836-e2d5a8c5f388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:08.395014 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a81f9202-4072-4959-9836-e2d5a8c5f388 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:08.396249 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a81f9202-4072-4959-9836-e2d5a8c5f388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:08.417418 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04625e7e-6dbc-4f0f-b027-805dd6e411f0 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:08.476842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60db85-25ba-443f-91c9-4e99b5b2a1cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69d216a568b546589e854c2b4476efe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:08.477579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60db85-25ba-443f-91c9-4e99b5b2a1cb service glance] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:08.477958 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60db85-25ba-443f-91c9-4e99b5b2a1cb service glance] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:08.478310 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60db85-25ba-443f-91c9-4e99b5b2a1cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:08.480238 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9d60db85-25ba-443f-91c9-4e99b5b2a1cb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:08.481818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1238/2512] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:08.482897 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1572e0e-6e89-48e5-bc16-a4f6c4825add None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:08.483226 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1572e0e-6e89-48e5-bc16-a4f6c4825add None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:08.483503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1572e0e-6e89-48e5-bc16-a4f6c4825add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:08.507224 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d1572e0e-6e89-48e5-bc16-a4f6c4825add None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:08.508961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d1572e0e-6e89-48e5-bc16-a4f6c4825add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:08.517426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1239/2513] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:08.519243 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f091db-d713-4ad1-98b0-0e2872f3fc69 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:08.519527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f091db-d713-4ad1-98b0-0e2872f3fc69 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:08.519800 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f091db-d713-4ad1-98b0-0e2872f3fc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:08.541162 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f6f091db-d713-4ad1-98b0-0e2872f3fc69 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:08.542393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f6f091db-d713-4ad1-98b0-0e2872f3fc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:08.546839 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60db85-25ba-443f-91c9-4e99b5b2a1cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:08.548293 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60db85-25ba-443f-91c9-4e99b5b2a1cb service glance] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:08.551918 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1240/2514] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:08.552606 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f091db-d713-4ad1-98b0-0e2872f3fc69 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:08.553874 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f6f091db-d713-4ad1-98b0-0e2872f3fc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:08.563309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1275/2515] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:08.564164 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60db85-25ba-443f-91c9-4e99b5b2a1cb service glance] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:08.587558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60db85-25ba-443f-91c9-4e99b5b2a1cb service glance] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:08.593931 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4257f-e229-4ed9-aa7e-cc339e1c1981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:08.595473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4257f-e229-4ed9-aa7e-cc339e1c1981 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:08.595753 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4257f-e229-4ed9-aa7e-cc339e1c1981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:08.596673 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4257f-e229-4ed9-aa7e-cc339e1c1981 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:08.597440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c4257f-e229-4ed9-aa7e-cc339e1c1981 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:09:08.597741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c4257f-e229-4ed9-aa7e-cc339e1c1981 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:08.600393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c4257f-e229-4ed9-aa7e-cc339e1c1981 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:08.620581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1276/2516] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:09:08.622716 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4257f-e229-4ed9-aa7e-cc339e1c1981 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:08.624071 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f091db-d713-4ad1-98b0-0e2872f3fc69 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:08.627839 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f9bbc-5a25-4910-ad88-53f7c2cefdbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:08.628472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f9bbc-5a25-4910-ad88-53f7c2cefdbc service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:08.628774 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f9bbc-5a25-4910-ad88-53f7c2cefdbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:08.629038 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f9bbc-5a25-4910-ad88-53f7c2cefdbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:08.636532 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4257f-e229-4ed9-aa7e-cc339e1c1981 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:08.640877 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f9bbc-5a25-4910-ad88-53f7c2cefdbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:08.642422 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f9bbc-5a25-4910-ad88-53f7c2cefdbc service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:08.644829 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaea763-9f40-41c4-b208-223998a64684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:08.645543 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaea763-9f40-41c4-b208-223998a64684 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:08.645910 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaea763-9f40-41c4-b208-223998a64684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:08.646395 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaea763-9f40-41c4-b208-223998a64684 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:08.647081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaea763-9f40-41c4-b208-223998a64684 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:08.650324 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaea763-9f40-41c4-b208-223998a64684 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:08.655380 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1241/2517] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:08.679936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:08.691020 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1277/2518] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:08 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:08.694736 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f9bbc-5a25-4910-ad88-53f7c2cefdbc service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:08.706958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f9bbc-5a25-4910-ad88-53f7c2cefdbc service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:08.738821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:08.835812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73272bed-39dd-453b-bde3-68fa4f1b62a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:08.837174 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73272bed-39dd-453b-bde3-68fa4f1b62a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:08.837857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73272bed-39dd-453b-bde3-68fa4f1b62a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:08.838847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73272bed-39dd-453b-bde3-68fa4f1b62a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:08.840568 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73272bed-39dd-453b-bde3-68fa4f1b62a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:08.851500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73272bed-39dd-453b-bde3-68fa4f1b62a8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:08.900583 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1242/2519] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:08 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:09:08.904892 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaea763-9f40-41c4-b208-223998a64684 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:08.920577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaea763-9f40-41c4-b208-223998a64684 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:08.926470 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7b4db-dfa0-480e-8461-72d14eca581e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:08.927312 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7b4db-dfa0-480e-8461-72d14eca581e admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:08.927656 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7b4db-dfa0-480e-8461-72d14eca581e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:08.928079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7b4db-dfa0-480e-8461-72d14eca581e admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:08.928923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7b4db-dfa0-480e-8461-72d14eca581e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:08.930681 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7b4db-dfa0-480e-8461-72d14eca581e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:08.943108 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1278/2520] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:08.947891 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73272bed-39dd-453b-bde3-68fa4f1b62a8 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:08.962514 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73272bed-39dd-453b-bde3-68fa4f1b62a8 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:08.967932 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4fa6a-a0d4-4ead-940e-b2aaafeefadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:08.968573 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4fa6a-a0d4-4ead-940e-b2aaafeefadb admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:08.968864 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4fa6a-a0d4-4ead-940e-b2aaafeefadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:08.969143 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4fa6a-a0d4-4ead-940e-b2aaafeefadb admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:08.969931 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4fa6a-a0d4-4ead-940e-b2aaafeefadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:08.972116 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4fa6a-a0d4-4ead-940e-b2aaafeefadb admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:08.984604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1243/2521] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:09:08.990695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7b4db-dfa0-480e-8461-72d14eca581e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:09.006144 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7b4db-dfa0-480e-8461-72d14eca581e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:09.012755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa6a1a-2c9f-48bc-89ba-6944989e3bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:09.012755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa6a1a-2c9f-48bc-89ba-6944989e3bce admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:09.012755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa6a1a-2c9f-48bc-89ba-6944989e3bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:09.016448 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa6a1a-2c9f-48bc-89ba-6944989e3bce admin admin] PATH_INFO: `/v3/projects/daa1ccae25954cbcb122c5fe10baefe0/users/0d004ec91e72491fac61653b571f18a8/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:09.033819 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa6a1a-2c9f-48bc-89ba-6944989e3bce admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa1ccae25954cbcb122c5fe10baefe0, user_id=0d004ec91e72491fac61653b571f18a8, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:09.036607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa6a1a-2c9f-48bc-89ba-6944989e3bce admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:09.039156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4fa6a-a0d4-4ead-940e-b2aaafeefadb admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:09.041210 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b0d4fa6a-a0d4-4ead-940e-b2aaafeefadb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:09.070319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1279/2522] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:08 2022] PUT /identity/v3/projects/daa1ccae25954cbcb122c5fe10baefe0/users/0d004ec91e72491fac61653b571f18a8/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:09.082230 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4fa6a-a0d4-4ead-940e-b2aaafeefadb admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:09.082876 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337f417-1166-4b06-b5f9-ce4f90b52fa7 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:09.083148 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337f417-1166-4b06-b5f9-ce4f90b52fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:09.083419 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337f417-1166-4b06-b5f9-ce4f90b52fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:09.119897 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-6337f417-1166-4b06-b5f9-ce4f90b52fa7 None None] MFA Rules not processed for user `0d004ec91e72491fac61653b571f18a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:09.121559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6337f417-1166-4b06-b5f9-ce4f90b52fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:09.156151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b02b4-31d2-4879-8255-9994eea2a92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:09.156915 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b02b4-31d2-4879-8255-9994eea2a92a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:09.157271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b02b4-31d2-4879-8255-9994eea2a92a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:09.157620 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b02b4-31d2-4879-8255-9994eea2a92a admin admin] PATH_INFO: `/v3/projects/50802b0f362042ef982dcfb1ef5c3744` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:09.160233 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b02b4-31d2-4879-8255-9994eea2a92a admin admin] RBAC: Authorizing `identity:delete_project(project_id=50802b0f362042ef982dcfb1ef5c3744)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:09.162328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b02b4-31d2-4879-8255-9994eea2a92a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:09.251691 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1280/2523] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:09.258501 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337f417-1166-4b06-b5f9-ce4f90b52fa7 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:09.261496 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a73b02b4-31d2-4879-8255-9994eea2a92a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50802b0f362042ef982dcfb1ef5c3744', 'request_id': 'req-a73b02b4-31d2-4879-8255-9994eea2a92a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:09.271335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337f417-1166-4b06-b5f9-ce4f90b52fa7 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:09.273234 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a73b02b4-31d2-4879-8255-9994eea2a92a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50802b0f362042ef982dcfb1ef5c3744', 'request_id': 'req-a73b02b4-31d2-4879-8255-9994eea2a92a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:09.283012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a73b02b4-31d2-4879-8255-9994eea2a92a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50802b0f362042ef982dcfb1ef5c3744', 'request_id': 'req-a73b02b4-31d2-4879-8255-9994eea2a92a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:09.299909 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a73b02b4-31d2-4879-8255-9994eea2a92a admin admin] The token cache is being invalidate because project 50802b0f362042ef982dcfb1ef5c3744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:09.300190 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a73b02b4-31d2-4879-8255-9994eea2a92a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:09.301048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a73b02b4-31d2-4879-8255-9994eea2a92a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:09.302832 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1244/2524] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:09 2022] DELETE /identity/v3/projects/50802b0f362042ef982dcfb1ef5c3744 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:09.308768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd27499-14f1-4d51-8171-aac1315201b7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:09.309044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd27499-14f1-4d51-8171-aac1315201b7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:09.309311 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd27499-14f1-4d51-8171-aac1315201b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:09.330948 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-fcd27499-14f1-4d51-8171-aac1315201b7 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:09.332220 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-fcd27499-14f1-4d51-8171-aac1315201b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:09.342312 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20dc2c-0f77-484e-a163-bcb685b36f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:09.342969 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20dc2c-0f77-484e-a163-bcb685b36f24 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:09.343402 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20dc2c-0f77-484e-a163-bcb685b36f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:09.343683 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20dc2c-0f77-484e-a163-bcb685b36f24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:09.344579 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20dc2c-0f77-484e-a163-bcb685b36f24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:09.346626 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20dc2c-0f77-484e-a163-bcb685b36f24 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:09.365647 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:09.382143 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1281/2525] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:09 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:09:09.386807 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20dc2c-0f77-484e-a163-bcb685b36f24 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:09.388721 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1b20dc2c-0f77-484e-a163-bcb685b36f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:09.404998 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1245/2526] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:09:09.413225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617a790-1c5b-4371-88ec-af3e598332ef None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:09.413521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617a790-1c5b-4371-88ec-af3e598332ef None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:09.413802 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617a790-1c5b-4371-88ec-af3e598332ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:09.436347 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-1617a790-1c5b-4371-88ec-af3e598332ef None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:09.437646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1617a790-1c5b-4371-88ec-af3e598332ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:09.440613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:09.466064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20dc2c-0f77-484e-a163-bcb685b36f24 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:09.471986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900027ac-1d1f-49d8-b74a-0808e703ee03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:09.472782 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900027ac-1d1f-49d8-b74a-0808e703ee03 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:09.472899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900027ac-1d1f-49d8-b74a-0808e703ee03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:09.473160 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900027ac-1d1f-49d8-b74a-0808e703ee03 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:09.473930 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900027ac-1d1f-49d8-b74a-0808e703ee03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:09.475848 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900027ac-1d1f-49d8-b74a-0808e703ee03 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:09.502066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1282/2527] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:09 2022] POST /identity/v3/users => generated 294 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:09.506201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900027ac-1d1f-49d8-b74a-0808e703ee03 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:09.507422 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1246/2528] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:09:09.512353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617a790-1c5b-4371-88ec-af3e598332ef None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:09.517122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900027ac-1d1f-49d8-b74a-0808e703ee03 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:09.522926 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20859270-1eb1-42f8-b5f6-7a753a2aea1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:09.523137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617a790-1c5b-4371-88ec-af3e598332ef None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:09.524653 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20859270-1eb1-42f8-b5f6-7a753a2aea1a admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:09.524653 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20859270-1eb1-42f8-b5f6-7a753a2aea1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:09.524653 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20859270-1eb1-42f8-b5f6-7a753a2aea1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:09.525558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20859270-1eb1-42f8-b5f6-7a753a2aea1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:09.527238 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20859270-1eb1-42f8-b5f6-7a753a2aea1a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:09.529047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11869b-5ae2-4755-980e-d98aefbf9159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:09.529989 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11869b-5ae2-4755-980e-d98aefbf9159 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:09.530379 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11869b-5ae2-4755-980e-d98aefbf9159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:09.530773 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11869b-5ae2-4755-980e-d98aefbf9159 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:09.531586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11869b-5ae2-4755-980e-d98aefbf9159 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:09:09.531993 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11869b-5ae2-4755-980e-d98aefbf9159 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:09.534671 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11869b-5ae2-4755-980e-d98aefbf9159 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:09.538853 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1283/2529] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:09:09.543617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20859270-1eb1-42f8-b5f6-7a753a2aea1a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:09.554081 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1247/2530] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:09:09.556503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11869b-5ae2-4755-980e-d98aefbf9159 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:09.558202 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20859270-1eb1-42f8-b5f6-7a753a2aea1a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:09.563267 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b120c-9c4a-46d3-8b17-efc594816b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:09.563985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b120c-9c4a-46d3-8b17-efc594816b37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:09.564878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b120c-9c4a-46d3-8b17-efc594816b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:09.565156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b120c-9c4a-46d3-8b17-efc594816b37 admin admin] PATH_INFO: `/v3/projects/1a7f95a3a93e4ad6a1c1c2dc54feddad/users/4888f55c998f4cb7b027f12e54b75cfa/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:09.568453 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11869b-5ae2-4755-980e-d98aefbf9159 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:09.574830 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d20b3-e468-4709-8639-eac256e29d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:09.575481 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d20b3-e468-4709-8639-eac256e29d7c admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:09.575823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d20b3-e468-4709-8639-eac256e29d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:09.576110 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d20b3-e468-4709-8639-eac256e29d7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:09.576780 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34d20b3-e468-4709-8639-eac256e29d7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:09.578805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34d20b3-e468-4709-8639-eac256e29d7c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:09.587901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b120c-9c4a-46d3-8b17-efc594816b37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a7f95a3a93e4ad6a1c1c2dc54feddad, user_id=4888f55c998f4cb7b027f12e54b75cfa, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:09.591062 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b120c-9c4a-46d3-8b17-efc594816b37 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:09.611521 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1284/2531] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:09 2022] PUT /identity/v3/projects/1a7f95a3a93e4ad6a1c1c2dc54feddad/users/4888f55c998f4cb7b027f12e54b75cfa/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:09.620854 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b120c-9c4a-46d3-8b17-efc594816b37 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:09.625149 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1248/2532] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:09 2022] POST /identity/v3/projects => generated 365 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:09:09.630979 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d20b3-e468-4709-8639-eac256e29d7c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:09.704058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b120c-9c4a-46d3-8b17-efc594816b37 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:09.711186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d20b3-e468-4709-8639-eac256e29d7c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:09.711792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2678da-b118-4625-86c0-35716f3ef8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:09.712167 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2678da-b118-4625-86c0-35716f3ef8ab admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:09.712511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2678da-b118-4625-86c0-35716f3ef8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:09.712888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2678da-b118-4625-86c0-35716f3ef8ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:09.713781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2678da-b118-4625-86c0-35716f3ef8ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:09.715597 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2678da-b118-4625-86c0-35716f3ef8ab admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:09.718432 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da487948-e6e4-4bb2-aa5f-29d3861ce55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:09.719086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da487948-e6e4-4bb2-aa5f-29d3861ce55a admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:09.719381 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da487948-e6e4-4bb2-aa5f-29d3861ce55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:09.719656 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da487948-e6e4-4bb2-aa5f-29d3861ce55a admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:09.720334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da487948-e6e4-4bb2-aa5f-29d3861ce55a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:09.722389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da487948-e6e4-4bb2-aa5f-29d3861ce55a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:09.727590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1285/2533] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:09.731094 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2678da-b118-4625-86c0-35716f3ef8ab admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:09.742430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2678da-b118-4625-86c0-35716f3ef8ab admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:09.747215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448689a-8e62-43f4-8cd9-faab040d0768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:09.748673 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1249/2534] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:09 2022] POST /identity/v3/users => generated 283 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:09.748841 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448689a-8e62-43f4-8cd9-faab040d0768 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:09.748841 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448689a-8e62-43f4-8cd9-faab040d0768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:09.748841 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448689a-8e62-43f4-8cd9-faab040d0768 admin admin] PATH_INFO: `/v3/projects/1a7f95a3a93e4ad6a1c1c2dc54feddad/users/4888f55c998f4cb7b027f12e54b75cfa/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:09.752533 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da487948-e6e4-4bb2-aa5f-29d3861ce55a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:09.753205 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448689a-8e62-43f4-8cd9-faab040d0768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a7f95a3a93e4ad6a1c1c2dc54feddad, user_id=4888f55c998f4cb7b027f12e54b75cfa, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:09.755949 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448689a-8e62-43f4-8cd9-faab040d0768 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:09.763534 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da487948-e6e4-4bb2-aa5f-29d3861ce55a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:09.769912 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1286/2535] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:09 2022] PUT /identity/v3/projects/1a7f95a3a93e4ad6a1c1c2dc54feddad/users/4888f55c998f4cb7b027f12e54b75cfa/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:09.782279 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4843634-9df4-48cb-bf97-af3dd1b17a2b None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:09.782577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4843634-9df4-48cb-bf97-af3dd1b17a2b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:09.782890 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4843634-9df4-48cb-bf97-af3dd1b17a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:09.799972 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:09.825245 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f4843634-9df4-48cb-bf97-af3dd1b17a2b None None] MFA Rules not processed for user `4888f55c998f4cb7b027f12e54b75cfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:09.826449 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f4843634-9df4-48cb-bf97-af3dd1b17a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:09.843826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21dbb5-ea5b-46d8-b7fb-52d6d14a8fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:09.843826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21dbb5-ea5b-46d8-b7fb-52d6d14a8fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:09.843826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21dbb5-ea5b-46d8-b7fb-52d6d14a8fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:09.843826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21dbb5-ea5b-46d8-b7fb-52d6d14a8fe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:09.844533 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21dbb5-ea5b-46d8-b7fb-52d6d14a8fe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:09.845844 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21dbb5-ea5b-46d8-b7fb-52d6d14a8fe2 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:09.859979 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1250/2536] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:09:09.865257 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21dbb5-ea5b-46d8-b7fb-52d6d14a8fe2 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:09.874950 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21dbb5-ea5b-46d8-b7fb-52d6d14a8fe2 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:09.880328 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca3358-d1fb-48ab-af3f-810747498db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:09.880955 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca3358-d1fb-48ab-af3f-810747498db6 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:09.881265 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca3358-d1fb-48ab-af3f-810747498db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:09.881545 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca3358-d1fb-48ab-af3f-810747498db6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:09.882280 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca3358-d1fb-48ab-af3f-810747498db6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:09.884089 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca3358-d1fb-48ab-af3f-810747498db6 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:09.900079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1251/2537] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:09.901278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca3358-d1fb-48ab-af3f-810747498db6 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:09.915570 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca3358-d1fb-48ab-af3f-810747498db6 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:09.929048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377cd3ea-a274-45b4-a6f9-988f168e1c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:09.929738 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cd3ea-a274-45b4-a6f9-988f168e1c54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:09.930153 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cd3ea-a274-45b4-a6f9-988f168e1c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:09.930433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cd3ea-a274-45b4-a6f9-988f168e1c54 admin admin] PATH_INFO: `/v3/projects/a2c1c2943e374e9f868b864ca95b0804/users/0512c530279442b1a58a0ab7894b5c1d/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:09.931446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1287/2538] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:09.946258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4843634-9df4-48cb-bf97-af3dd1b17a2b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:09.948292 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f4843634-9df4-48cb-bf97-af3dd1b17a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:09.954180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:09.964255 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377cd3ea-a274-45b4-a6f9-988f168e1c54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2c1c2943e374e9f868b864ca95b0804, user_id=0512c530279442b1a58a0ab7894b5c1d, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:09.969673 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377cd3ea-a274-45b4-a6f9-988f168e1c54 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:09.987940 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1252/2539] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:09 2022] PUT /identity/v3/projects/a2c1c2943e374e9f868b864ca95b0804/users/0512c530279442b1a58a0ab7894b5c1d/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:10.005754 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8b176-9815-4306-abde-2e228db289e4 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:10.006329 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8b176-9815-4306-abde-2e228db289e4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:10.006720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8b176-9815-4306-abde-2e228db289e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:10.038194 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4843634-9df4-48cb-bf97-af3dd1b17a2b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:10.041663 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7bb8b176-9815-4306-abde-2e228db289e4 None None] MFA Rules not processed for user `0512c530279442b1a58a0ab7894b5c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:10.043703 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7bb8b176-9815-4306-abde-2e228db289e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:10.106539 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c714f-e490-443b-b6ae-108fd82c3565 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:10.107177 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c714f-e490-443b-b6ae-108fd82c3565 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:10.107464 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c714f-e490-443b-b6ae-108fd82c3565 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:10.107734 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c714f-e490-443b-b6ae-108fd82c3565 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:10.109629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-364c714f-e490-443b-b6ae-108fd82c3565 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:10.145868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1253/2540] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:09:10.157967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8b176-9815-4306-abde-2e228db289e4 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:10.159604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7bb8b176-9815-4306-abde-2e228db289e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:10.177917 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8b176-9815-4306-abde-2e228db289e4 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:10.183318 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956718b-38ab-43c4-a26f-54609002bc3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:10.184024 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956718b-38ab-43c4-a26f-54609002bc3d service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:10.184325 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956718b-38ab-43c4-a26f-54609002bc3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:10.184598 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956718b-38ab-43c4-a26f-54609002bc3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:10.188958 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c714f-e490-443b-b6ae-108fd82c3565 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:10.190984 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c714f-e490-443b-b6ae-108fd82c3565 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:10.195070 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956718b-38ab-43c4-a26f-54609002bc3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:10.196485 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956718b-38ab-43c4-a26f-54609002bc3d service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:10.200995 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1288/2541] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:10.209975 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1254/2542] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:10 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:17.039056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c714f-e490-443b-b6ae-108fd82c3565 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:17.053983 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c714f-e490-443b-b6ae-108fd82c3565 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:17.058049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9bee6-fb81-4aa5-9a8f-bec7189319e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:17.058677 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9bee6-fb81-4aa5-9a8f-bec7189319e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:17.058959 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9bee6-fb81-4aa5-9a8f-bec7189319e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:17.059235 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9bee6-fb81-4aa5-9a8f-bec7189319e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:17.125323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a9bee6-fb81-4aa5-9a8f-bec7189319e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:17.126826 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a9bee6-fb81-4aa5-9a8f-bec7189319e9 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:17.137167 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1289/2543] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:17 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:20.201822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=69,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:27.132461 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:28.445445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956718b-38ab-43c4-a26f-54609002bc3d service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:28.538522 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956718b-38ab-43c4-a26f-54609002bc3d service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:28.544295 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4fd83-0a45-4cb1-8e04-227696cc9862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:28.544801 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4fd83-0a45-4cb1-8e04-227696cc9862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:28.545173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4fd83-0a45-4cb1-8e04-227696cc9862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:28.545477 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4fd83-0a45-4cb1-8e04-227696cc9862 admin admin] PATH_INFO: `/v3/users/0512c530279442b1a58a0ab7894b5c1d` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:28.546942 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4fd83-0a45-4cb1-8e04-227696cc9862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0512c530279442b1a58a0ab7894b5c1d)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:28.549153 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4fd83-0a45-4cb1-8e04-227696cc9862 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:28.623203 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-79d4fd83-0a45-4cb1-8e04-227696cc9862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0512c530279442b1a58a0ab7894b5c1d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:28.623881 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-79d4fd83-0a45-4cb1-8e04-227696cc9862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0512c530279442b1a58a0ab7894b5c1d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:28.633581 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-79d4fd83-0a45-4cb1-8e04-227696cc9862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0512c530279442b1a58a0ab7894b5c1d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:28.649724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-79d4fd83-0a45-4cb1-8e04-227696cc9862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0512c530279442b1a58a0ab7894b5c1d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:28.652086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-79d4fd83-0a45-4cb1-8e04-227696cc9862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0512c530279442b1a58a0ab7894b5c1d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:28.666610 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1255/2544] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:28 2022] DELETE /identity/v3/users/0512c530279442b1a58a0ab7894b5c1d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:28.671903 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4613e-d202-4e15-bb9c-4f01485d40f8 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:28.672425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4613e-d202-4e15-bb9c-4f01485d40f8 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:28.672765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4613e-d202-4e15-bb9c-4f01485d40f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:28.695444 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-aef4613e-d202-4e15-bb9c-4f01485d40f8 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:28.696913 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-aef4613e-d202-4e15-bb9c-4f01485d40f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:28.770686 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1290/2545] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:28.788024 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4fd83-0a45-4cb1-8e04-227696cc9862 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:28.789510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-79d4fd83-0a45-4cb1-8e04-227696cc9862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:28.856883 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4fd83-0a45-4cb1-8e04-227696cc9862 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:28.860641 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91acf2-8782-439e-9795-fb2b9c9bdce7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:28.861329 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91acf2-8782-439e-9795-fb2b9c9bdce7 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:28.862246 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91acf2-8782-439e-9795-fb2b9c9bdce7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:28.863034 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91acf2-8782-439e-9795-fb2b9c9bdce7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:28.874154 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91acf2-8782-439e-9795-fb2b9c9bdce7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:28.875618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91acf2-8782-439e-9795-fb2b9c9bdce7 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:28.887325 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1256/2546] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:29.168048 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4613e-d202-4e15-bb9c-4f01485d40f8 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:29.169487 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-aef4613e-d202-4e15-bb9c-4f01485d40f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:29.183172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4613e-d202-4e15-bb9c-4f01485d40f8 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:29.188222 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b5054-1d97-4aa8-aca9-85e915f21c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:29.188978 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b5054-1d97-4aa8-aca9-85e915f21c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:29.189930 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b5054-1d97-4aa8-aca9-85e915f21c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:29.190245 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b5054-1d97-4aa8-aca9-85e915f21c23 admin admin] PATH_INFO: `/v3/projects/a2c1c2943e374e9f868b864ca95b0804` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:29.191511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b5054-1d97-4aa8-aca9-85e915f21c23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2c1c2943e374e9f868b864ca95b0804)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:29.193391 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b5054-1d97-4aa8-aca9-85e915f21c23 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:29.241257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c80b5054-1d97-4aa8-aca9-85e915f21c23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2c1c2943e374e9f868b864ca95b0804', 'request_id': 'req-c80b5054-1d97-4aa8-aca9-85e915f21c23'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:29.246508 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c80b5054-1d97-4aa8-aca9-85e915f21c23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2c1c2943e374e9f868b864ca95b0804', 'request_id': 'req-c80b5054-1d97-4aa8-aca9-85e915f21c23'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:29.253106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c80b5054-1d97-4aa8-aca9-85e915f21c23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2c1c2943e374e9f868b864ca95b0804', 'request_id': 'req-c80b5054-1d97-4aa8-aca9-85e915f21c23'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:29.268724 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c80b5054-1d97-4aa8-aca9-85e915f21c23 admin admin] The token cache is being invalidate because project a2c1c2943e374e9f868b864ca95b0804 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:29.269015 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c80b5054-1d97-4aa8-aca9-85e915f21c23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:29.269645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c80b5054-1d97-4aa8-aca9-85e915f21c23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:29.271197 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1291/2547] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:29 2022] DELETE /identity/v3/projects/a2c1c2943e374e9f868b864ca95b0804 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:29.276302 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc676b-96c2-450a-89b3-596dce7c6ead None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:29.276586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc676b-96c2-450a-89b3-596dce7c6ead None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:29.276861 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc676b-96c2-450a-89b3-596dce7c6ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:29.296720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-b5bc676b-96c2-450a-89b3-596dce7c6ead None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:29.297985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b5bc676b-96c2-450a-89b3-596dce7c6ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:29.317503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:29.365827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1257/2548] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:29.373737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ade1f6-6bce-403e-83de-bed625e07547 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:29.374067 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ade1f6-6bce-403e-83de-bed625e07547 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:29.374347 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ade1f6-6bce-403e-83de-bed625e07547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:29.396697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-16ade1f6-6bce-403e-83de-bed625e07547 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:29.397982 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-16ade1f6-6bce-403e-83de-bed625e07547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:29.406230 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1292/2549] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:29.413907 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc676b-96c2-450a-89b3-596dce7c6ead None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:29.424392 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc676b-96c2-450a-89b3-596dce7c6ead None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:29.429256 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6b55e-cc1a-43eb-893c-290114b51b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:29.429909 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6b55e-cc1a-43eb-893c-290114b51b87 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:29.430195 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6b55e-cc1a-43eb-893c-290114b51b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:29.430470 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6b55e-cc1a-43eb-893c-290114b51b87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:29.431166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6b55e-cc1a-43eb-893c-290114b51b87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:09:29.431474 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6b55e-cc1a-43eb-893c-290114b51b87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:29.433090 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6b55e-cc1a-43eb-893c-290114b51b87 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:29.445947 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1258/2550] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:09:29.450559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ade1f6-6bce-403e-83de-bed625e07547 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:29.460455 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ade1f6-6bce-403e-83de-bed625e07547 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:29.467347 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8e73d-c36c-43f4-b4b9-ed4e9ebd013a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:29.468328 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8e73d-c36c-43f4-b4b9-ed4e9ebd013a admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:29.468816 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8e73d-c36c-43f4-b4b9-ed4e9ebd013a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:29.469252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8e73d-c36c-43f4-b4b9-ed4e9ebd013a admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:29.470113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8e73d-c36c-43f4-b4b9-ed4e9ebd013a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:29.472528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8e73d-c36c-43f4-b4b9-ed4e9ebd013a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:29.505380 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1293/2551] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:29 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:29.509760 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6b55e-cc1a-43eb-893c-290114b51b87 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:29.520079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6b55e-cc1a-43eb-893c-290114b51b87 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:29.592011 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73070447-1cf4-4337-a4e7-cfb023166919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:29.592662 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73070447-1cf4-4337-a4e7-cfb023166919 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:29.592944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73070447-1cf4-4337-a4e7-cfb023166919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:29.593221 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73070447-1cf4-4337-a4e7-cfb023166919 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:29.593925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73070447-1cf4-4337-a4e7-cfb023166919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:29.596514 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73070447-1cf4-4337-a4e7-cfb023166919 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:29.619081 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1259/2552] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:29 2022] POST /identity/v3/users => generated 282 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:29.623244 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8e73d-c36c-43f4-b4b9-ed4e9ebd013a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:29.634072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8e73d-c36c-43f4-b4b9-ed4e9ebd013a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:29.639123 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78536c13-a21f-477f-823d-d37fae8ea4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:29.639751 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78536c13-a21f-477f-823d-d37fae8ea4bd admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:29.640030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78536c13-a21f-477f-823d-d37fae8ea4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:29.640299 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78536c13-a21f-477f-823d-d37fae8ea4bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:29.641018 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78536c13-a21f-477f-823d-d37fae8ea4bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:29.642760 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78536c13-a21f-477f-823d-d37fae8ea4bd admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:29.659355 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1294/2553] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:29.663424 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73070447-1cf4-4337-a4e7-cfb023166919 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:29.676334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73070447-1cf4-4337-a4e7-cfb023166919 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:29.681982 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aaa531-3f9d-41b8-b7aa-edcad8677664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:29.682612 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aaa531-3f9d-41b8-b7aa-edcad8677664 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:29.682898 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aaa531-3f9d-41b8-b7aa-edcad8677664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:29.683176 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aaa531-3f9d-41b8-b7aa-edcad8677664 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:29.683877 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aaa531-3f9d-41b8-b7aa-edcad8677664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:29.685498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aaa531-3f9d-41b8-b7aa-edcad8677664 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:29.697171 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1260/2554] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:29.701487 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78536c13-a21f-477f-823d-d37fae8ea4bd admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:29.712149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78536c13-a21f-477f-823d-d37fae8ea4bd admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:29.717044 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efce296-ce73-4bc0-b883-cc36dfaec803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:29.717713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efce296-ce73-4bc0-b883-cc36dfaec803 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:29.718029 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efce296-ce73-4bc0-b883-cc36dfaec803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:29.718309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efce296-ce73-4bc0-b883-cc36dfaec803 admin admin] PATH_INFO: `/v3/projects/7738fd6bcde74a72847e8ee306951d29/users/a6376a80f6694b149a4cde5e2be7b4bb/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:29.734294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efce296-ce73-4bc0-b883-cc36dfaec803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7738fd6bcde74a72847e8ee306951d29, user_id=a6376a80f6694b149a4cde5e2be7b4bb, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:29.737792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efce296-ce73-4bc0-b883-cc36dfaec803 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:29.755201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1295/2555] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:29 2022] PUT /identity/v3/projects/7738fd6bcde74a72847e8ee306951d29/users/a6376a80f6694b149a4cde5e2be7b4bb/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:29.765686 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1837e70-27aa-4e40-8882-f2cb9711096b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:29.765918 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1837e70-27aa-4e40-8882-f2cb9711096b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:29.766214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1837e70-27aa-4e40-8882-f2cb9711096b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:29.795891 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f1837e70-27aa-4e40-8882-f2cb9711096b None None] MFA Rules not processed for user `a6376a80f6694b149a4cde5e2be7b4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:29.797129 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f1837e70-27aa-4e40-8882-f2cb9711096b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:29.868705 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1261/2556] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:29 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:29.872923 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efce296-ce73-4bc0-b883-cc36dfaec803 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:29.895775 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:29.943977 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efce296-ce73-4bc0-b883-cc36dfaec803 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:29.948800 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86de991-848c-404c-8d13-8daef81d5670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:29.949424 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86de991-848c-404c-8d13-8daef81d5670 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:29.949706 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86de991-848c-404c-8d13-8daef81d5670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:29.950014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86de991-848c-404c-8d13-8daef81d5670 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:29.950684 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86de991-848c-404c-8d13-8daef81d5670 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:29.952580 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86de991-848c-404c-8d13-8daef81d5670 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:29.981208 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1296/2557] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:29 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:29.985363 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1837e70-27aa-4e40-8882-f2cb9711096b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:29.996527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1837e70-27aa-4e40-8882-f2cb9711096b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:30.009325 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:30.064784 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3353a-8cba-404b-a6b3-70d69b9097f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:30.065451 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3353a-8cba-404b-a6b3-70d69b9097f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:30.065736 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3353a-8cba-404b-a6b3-70d69b9097f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:30.066058 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3353a-8cba-404b-a6b3-70d69b9097f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:30.066746 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3353a-8cba-404b-a6b3-70d69b9097f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:30.068816 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3353a-8cba-404b-a6b3-70d69b9097f4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:30.092704 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1262/2558] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:29 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:30.097247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86de991-848c-404c-8d13-8daef81d5670 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:30.107643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86de991-848c-404c-8d13-8daef81d5670 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:30.113166 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaadcbd-15f8-4311-b185-9714aa3e3efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:30.113867 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaadcbd-15f8-4311-b185-9714aa3e3efb admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:30.114285 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaadcbd-15f8-4311-b185-9714aa3e3efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:30.114772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaadcbd-15f8-4311-b185-9714aa3e3efb admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:30.115983 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaadcbd-15f8-4311-b185-9714aa3e3efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:30.118658 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaadcbd-15f8-4311-b185-9714aa3e3efb admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:30.130390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1297/2559] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:30.134689 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3353a-8cba-404b-a6b3-70d69b9097f4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:30.145160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3353a-8cba-404b-a6b3-70d69b9097f4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:30.150013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8b29d-22e9-458f-a01a-e95151035473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:30.150686 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8b29d-22e9-458f-a01a-e95151035473 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:30.150978 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8b29d-22e9-458f-a01a-e95151035473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:30.151258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8b29d-22e9-458f-a01a-e95151035473 admin admin] PATH_INFO: `/v3/projects/7a87b2af57764080987c47877522ff8b/users/4d537a11adb8414786db5323cef1e494/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:30.167517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8b29d-22e9-458f-a01a-e95151035473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a87b2af57764080987c47877522ff8b, user_id=4d537a11adb8414786db5323cef1e494, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:30.170245 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8b29d-22e9-458f-a01a-e95151035473 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:30.184122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1263/2560] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:30 2022] PUT /identity/v3/projects/7a87b2af57764080987c47877522ff8b/users/4d537a11adb8414786db5323cef1e494/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:30.188243 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaadcbd-15f8-4311-b185-9714aa3e3efb admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:30.198239 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaadcbd-15f8-4311-b185-9714aa3e3efb admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:30.256940 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f6db4-9475-46d9-b9cd-b87fd90a2acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:30.257553 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f6db4-9475-46d9-b9cd-b87fd90a2acf admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:30.257830 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f6db4-9475-46d9-b9cd-b87fd90a2acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:30.258139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f6db4-9475-46d9-b9cd-b87fd90a2acf admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:30.258819 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f6db4-9475-46d9-b9cd-b87fd90a2acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:30.260394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f6db4-9475-46d9-b9cd-b87fd90a2acf admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:30.271677 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1298/2561] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:30.276109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8b29d-22e9-458f-a01a-e95151035473 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:30.286718 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8b29d-22e9-458f-a01a-e95151035473 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:30.291440 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02722f1-08be-422c-8eaa-ae4fa46f13bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:30.292119 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02722f1-08be-422c-8eaa-ae4fa46f13bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:30.292403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02722f1-08be-422c-8eaa-ae4fa46f13bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:30.292680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02722f1-08be-422c-8eaa-ae4fa46f13bf admin admin] PATH_INFO: `/v3/projects/7a87b2af57764080987c47877522ff8b/users/4d537a11adb8414786db5323cef1e494/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:30.294710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02722f1-08be-422c-8eaa-ae4fa46f13bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a87b2af57764080987c47877522ff8b, user_id=4d537a11adb8414786db5323cef1e494, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:30.297376 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02722f1-08be-422c-8eaa-ae4fa46f13bf admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:30.308374 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1264/2562] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:30 2022] PUT /identity/v3/projects/7a87b2af57764080987c47877522ff8b/users/4d537a11adb8414786db5323cef1e494/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:30.320938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce6520-57b8-4258-b292-3844d32db6fb None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:30.321221 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce6520-57b8-4258-b292-3844d32db6fb None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:30.321500 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce6520-57b8-4258-b292-3844d32db6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:30.353138 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-80ce6520-57b8-4258-b292-3844d32db6fb None None] MFA Rules not processed for user `4d537a11adb8414786db5323cef1e494`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:30.354389 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-80ce6520-57b8-4258-b292-3844d32db6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:30.422940 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1299/2563] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:30 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:30.434503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02722f1-08be-422c-8eaa-ae4fa46f13bf admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:30.436267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c02722f1-08be-422c-8eaa-ae4fa46f13bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:30.508507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02722f1-08be-422c-8eaa-ae4fa46f13bf admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:30.512457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b0f00-1b92-4699-8696-59e9b3a3b5a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:30.513085 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0f00-1b92-4699-8696-59e9b3a3b5a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:30.513367 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0f00-1b92-4699-8696-59e9b3a3b5a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:30.513642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0f00-1b92-4699-8696-59e9b3a3b5a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:30.524967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b0f00-1b92-4699-8696-59e9b3a3b5a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:30.526487 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b0f00-1b92-4699-8696-59e9b3a3b5a1 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:30.536532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1265/2564] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:30 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:30.989204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce6520-57b8-4258-b292-3844d32db6fb None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:31.013309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:31.066912 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce6520-57b8-4258-b292-3844d32db6fb None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:31.071770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39144c8b-533a-4d05-9e54-06d290fc9b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:31.072370 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39144c8b-533a-4d05-9e54-06d290fc9b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:31.072655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39144c8b-533a-4d05-9e54-06d290fc9b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:31.072936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39144c8b-533a-4d05-9e54-06d290fc9b21 admin admin] PATH_INFO: `/v3/users/a6376a80f6694b149a4cde5e2be7b4bb` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:31.074543 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39144c8b-533a-4d05-9e54-06d290fc9b21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6376a80f6694b149a4cde5e2be7b4bb)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:31.076496 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39144c8b-533a-4d05-9e54-06d290fc9b21 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:31.125282 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-39144c8b-533a-4d05-9e54-06d290fc9b21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6376a80f6694b149a4cde5e2be7b4bb'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:31.125971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-39144c8b-533a-4d05-9e54-06d290fc9b21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6376a80f6694b149a4cde5e2be7b4bb'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:31.139866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-39144c8b-533a-4d05-9e54-06d290fc9b21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6376a80f6694b149a4cde5e2be7b4bb'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:31.153683 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-39144c8b-533a-4d05-9e54-06d290fc9b21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6376a80f6694b149a4cde5e2be7b4bb'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:31.160569 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-39144c8b-533a-4d05-9e54-06d290fc9b21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6376a80f6694b149a4cde5e2be7b4bb'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:31.166706 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1300/2565] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:30 2022] DELETE /identity/v3/users/a6376a80f6694b149a4cde5e2be7b4bb => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:31.171392 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b5398-de9e-4890-83f7-bab42f0b2a39 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:31.171684 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b5398-de9e-4890-83f7-bab42f0b2a39 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:31.171969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b5398-de9e-4890-83f7-bab42f0b2a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:31.191810 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6a3b5398-de9e-4890-83f7-bab42f0b2a39 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:31.193022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6a3b5398-de9e-4890-83f7-bab42f0b2a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:31.250677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1266/2566] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:31.261640 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39144c8b-533a-4d05-9e54-06d290fc9b21 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:31.263306 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-39144c8b-533a-4d05-9e54-06d290fc9b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:31.321197 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39144c8b-533a-4d05-9e54-06d290fc9b21 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:31.325035 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d8257-16ab-47b1-b0b3-1c6cc3850200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:31.325599 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d8257-16ab-47b1-b0b3-1c6cc3850200 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:31.325914 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d8257-16ab-47b1-b0b3-1c6cc3850200 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:31.326182 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d8257-16ab-47b1-b0b3-1c6cc3850200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:31.337077 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d8257-16ab-47b1-b0b3-1c6cc3850200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:31.338625 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d8257-16ab-47b1-b0b3-1c6cc3850200 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:31.348880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1301/2567] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:31.642062 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b5398-de9e-4890-83f7-bab42f0b2a39 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:31.643723 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6a3b5398-de9e-4890-83f7-bab42f0b2a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:31.652533 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:31.659395 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b5398-de9e-4890-83f7-bab42f0b2a39 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:31.666192 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bacec4-1ace-4c05-b1a6-4adaf1d4e331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:31.667093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bacec4-1ace-4c05-b1a6-4adaf1d4e331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:31.667369 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bacec4-1ace-4c05-b1a6-4adaf1d4e331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:31.667805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bacec4-1ace-4c05-b1a6-4adaf1d4e331 admin admin] PATH_INFO: `/v3/projects/7738fd6bcde74a72847e8ee306951d29` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:31.669797 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bacec4-1ace-4c05-b1a6-4adaf1d4e331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7738fd6bcde74a72847e8ee306951d29)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:31.671934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bacec4-1ace-4c05-b1a6-4adaf1d4e331 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:31.728408 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a9bacec4-1ace-4c05-b1a6-4adaf1d4e331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7738fd6bcde74a72847e8ee306951d29', 'request_id': 'req-a9bacec4-1ace-4c05-b1a6-4adaf1d4e331'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:31.734702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a9bacec4-1ace-4c05-b1a6-4adaf1d4e331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7738fd6bcde74a72847e8ee306951d29', 'request_id': 'req-a9bacec4-1ace-4c05-b1a6-4adaf1d4e331'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:31.741237 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a9bacec4-1ace-4c05-b1a6-4adaf1d4e331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7738fd6bcde74a72847e8ee306951d29', 'request_id': 'req-a9bacec4-1ace-4c05-b1a6-4adaf1d4e331'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:31.754071 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a9bacec4-1ace-4c05-b1a6-4adaf1d4e331 admin admin] The token cache is being invalidate because project 7738fd6bcde74a72847e8ee306951d29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:31.754362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a9bacec4-1ace-4c05-b1a6-4adaf1d4e331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:31.754997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a9bacec4-1ace-4c05-b1a6-4adaf1d4e331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:31.756547 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1267/2568] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:31 2022] DELETE /identity/v3/projects/7738fd6bcde74a72847e8ee306951d29 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:31.760784 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d8257-16ab-47b1-b0b3-1c6cc3850200 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:31.771003 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d8257-16ab-47b1-b0b3-1c6cc3850200 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:31.806613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:31.832932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097e685-6c34-4114-9b25-9252e1d85be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:31.833700 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097e685-6c34-4114-9b25-9252e1d85be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:31.834035 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097e685-6c34-4114-9b25-9252e1d85be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:31.834308 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097e685-6c34-4114-9b25-9252e1d85be6 admin admin] PATH_INFO: `/v3/users/4d537a11adb8414786db5323cef1e494` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:31.835500 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097e685-6c34-4114-9b25-9252e1d85be6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d537a11adb8414786db5323cef1e494)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:31.837458 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097e685-6c34-4114-9b25-9252e1d85be6 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:31.888069 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d097e685-6c34-4114-9b25-9252e1d85be6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d537a11adb8414786db5323cef1e494'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:31.888625 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d097e685-6c34-4114-9b25-9252e1d85be6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d537a11adb8414786db5323cef1e494'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:31.899217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d097e685-6c34-4114-9b25-9252e1d85be6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d537a11adb8414786db5323cef1e494'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:31.917819 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d097e685-6c34-4114-9b25-9252e1d85be6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d537a11adb8414786db5323cef1e494'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:31.925490 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d097e685-6c34-4114-9b25-9252e1d85be6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d537a11adb8414786db5323cef1e494'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:31.931633 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1302/2569] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:31 2022] DELETE /identity/v3/users/4d537a11adb8414786db5323cef1e494 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:32.217007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bacec4-1ace-4c05-b1a6-4adaf1d4e331 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:32.218508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a9bacec4-1ace-4c05-b1a6-4adaf1d4e331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:32.234884 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bacec4-1ace-4c05-b1a6-4adaf1d4e331 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:32.294691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf8076-1097-4200-add7-149ad0e1296d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:32.295348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf8076-1097-4200-add7-149ad0e1296d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:32.295634 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf8076-1097-4200-add7-149ad0e1296d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:32.295912 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf8076-1097-4200-add7-149ad0e1296d admin admin] PATH_INFO: `/v3/projects/7a87b2af57764080987c47877522ff8b` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:32.297133 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf8076-1097-4200-add7-149ad0e1296d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a87b2af57764080987c47877522ff8b)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:32.299047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf8076-1097-4200-add7-149ad0e1296d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:32.357814 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-19cf8076-1097-4200-add7-149ad0e1296d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a87b2af57764080987c47877522ff8b', 'request_id': 'req-19cf8076-1097-4200-add7-149ad0e1296d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:32.364185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-19cf8076-1097-4200-add7-149ad0e1296d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a87b2af57764080987c47877522ff8b', 'request_id': 'req-19cf8076-1097-4200-add7-149ad0e1296d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:32.369727 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-19cf8076-1097-4200-add7-149ad0e1296d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a87b2af57764080987c47877522ff8b', 'request_id': 'req-19cf8076-1097-4200-add7-149ad0e1296d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:32.382116 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-19cf8076-1097-4200-add7-149ad0e1296d admin admin] The token cache is being invalidate because project 7a87b2af57764080987c47877522ff8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:32.382389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-19cf8076-1097-4200-add7-149ad0e1296d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:32.384433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-19cf8076-1097-4200-add7-149ad0e1296d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:32.384675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1268/2570] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:32 2022] DELETE /identity/v3/projects/7a87b2af57764080987c47877522ff8b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:32.390586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51138a-48aa-41b8-b095-fcd9983d695d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:32.391088 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51138a-48aa-41b8-b095-fcd9983d695d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:32.391215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51138a-48aa-41b8-b095-fcd9983d695d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:32.415569 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-bb51138a-48aa-41b8-b095-fcd9983d695d None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:32.417113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-bb51138a-48aa-41b8-b095-fcd9983d695d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:32.482064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1303/2571] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:32.490455 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae8582-9472-4cff-8097-6485def5395c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:32.490741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae8582-9472-4cff-8097-6485def5395c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:32.491058 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae8582-9472-4cff-8097-6485def5395c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:32.511910 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-5bae8582-9472-4cff-8097-6485def5395c None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:32.513141 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5bae8582-9472-4cff-8097-6485def5395c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:32.521836 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1269/2572] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:09:32.526311 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51138a-48aa-41b8-b095-fcd9983d695d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:32.537073 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51138a-48aa-41b8-b095-fcd9983d695d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:32.541822 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae957333-dc9e-4c25-85d9-beb518fd9b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:32.542484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae957333-dc9e-4c25-85d9-beb518fd9b3e admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:32.542768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae957333-dc9e-4c25-85d9-beb518fd9b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:32.543046 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae957333-dc9e-4c25-85d9-beb518fd9b3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:32.543750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae957333-dc9e-4c25-85d9-beb518fd9b3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:09:32.544058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae957333-dc9e-4c25-85d9-beb518fd9b3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:32.545677 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae957333-dc9e-4c25-85d9-beb518fd9b3e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:32.557998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1304/2573] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:09:32.562475 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae8582-9472-4cff-8097-6485def5395c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:32.572262 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae8582-9472-4cff-8097-6485def5395c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:32.576830 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fc656-4ea1-49c6-86ca-e4e0d4f73c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:32.577455 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fc656-4ea1-49c6-86ca-e4e0d4f73c2e admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:32.577714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fc656-4ea1-49c6-86ca-e4e0d4f73c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:32.578018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fc656-4ea1-49c6-86ca-e4e0d4f73c2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:32.578682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fc656-4ea1-49c6-86ca-e4e0d4f73c2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:32.581333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fc656-4ea1-49c6-86ca-e4e0d4f73c2e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:32.586789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:32.610382 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1270/2574] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:32 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:09:32.614801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae957333-dc9e-4c25-85d9-beb518fd9b3e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:32.625217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae957333-dc9e-4c25-85d9-beb518fd9b3e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:32.669298 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:32.690683 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884b60b-97a7-44bf-be45-6116f8b395b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:32.691317 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884b60b-97a7-44bf-be45-6116f8b395b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:32.691628 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884b60b-97a7-44bf-be45-6116f8b395b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:32.691901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884b60b-97a7-44bf-be45-6116f8b395b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:32.692577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9884b60b-97a7-44bf-be45-6116f8b395b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:32.694573 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9884b60b-97a7-44bf-be45-6116f8b395b3 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:32.717468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1305/2575] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:32 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:09:32.722916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fc656-4ea1-49c6-86ca-e4e0d4f73c2e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:32.733503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fc656-4ea1-49c6-86ca-e4e0d4f73c2e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:32.739189 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40152ed1-bb8c-4634-9d5b-7db2ade58eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:32.739823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40152ed1-bb8c-4634-9d5b-7db2ade58eee admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:32.740114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40152ed1-bb8c-4634-9d5b-7db2ade58eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:32.740398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40152ed1-bb8c-4634-9d5b-7db2ade58eee admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:32.741132 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40152ed1-bb8c-4634-9d5b-7db2ade58eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:32.742906 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40152ed1-bb8c-4634-9d5b-7db2ade58eee admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:32.755845 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1271/2576] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:32.760176 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884b60b-97a7-44bf-be45-6116f8b395b3 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:32.772066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884b60b-97a7-44bf-be45-6116f8b395b3 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:32.777261 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5521f-ff65-4681-bc8c-e12df98e3ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:32.777911 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5521f-ff65-4681-bc8c-e12df98e3ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:32.778214 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5521f-ff65-4681-bc8c-e12df98e3ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:32.778486 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5521f-ff65-4681-bc8c-e12df98e3ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:32.779232 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5521f-ff65-4681-bc8c-e12df98e3ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:32.780783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5521f-ff65-4681-bc8c-e12df98e3ef3 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:32.795893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1306/2577] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:09:32.800160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40152ed1-bb8c-4634-9d5b-7db2ade58eee admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:32.810742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40152ed1-bb8c-4634-9d5b-7db2ade58eee admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:32.817558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413e2b8-dc4b-4b9a-ad1a-08a69ec0155b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:32.818274 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413e2b8-dc4b-4b9a-ad1a-08a69ec0155b admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:32.818560 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413e2b8-dc4b-4b9a-ad1a-08a69ec0155b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:32.818837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413e2b8-dc4b-4b9a-ad1a-08a69ec0155b admin admin] PATH_INFO: `/v3/projects/057c1d6ce34c42d3af74548086a3e64c/users/df1fa22d85c349adbd72bb0cfcb31bd9/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:32.835945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413e2b8-dc4b-4b9a-ad1a-08a69ec0155b admin admin] RBAC: Authorizing `identity:create_grant(project_id=057c1d6ce34c42d3af74548086a3e64c, user_id=df1fa22d85c349adbd72bb0cfcb31bd9, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:32.838678 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413e2b8-dc4b-4b9a-ad1a-08a69ec0155b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:32.853276 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1272/2578] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:32 2022] PUT /identity/v3/projects/057c1d6ce34c42d3af74548086a3e64c/users/df1fa22d85c349adbd72bb0cfcb31bd9/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:32.864695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692b127-962d-4a4f-9c5a-2ea1a31173f2 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:32.865033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692b127-962d-4a4f-9c5a-2ea1a31173f2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:32.865411 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692b127-962d-4a4f-9c5a-2ea1a31173f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:32.897255 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a692b127-962d-4a4f-9c5a-2ea1a31173f2 None None] MFA Rules not processed for user `df1fa22d85c349adbd72bb0cfcb31bd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:32.898553 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a692b127-962d-4a4f-9c5a-2ea1a31173f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:32.970922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1307/2579] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:32.975418 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413e2b8-dc4b-4b9a-ad1a-08a69ec0155b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:33.038843 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413e2b8-dc4b-4b9a-ad1a-08a69ec0155b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:33.043550 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37c554-5899-46ce-aebe-ea12c2049fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:33.044167 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37c554-5899-46ce-aebe-ea12c2049fb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:33.044456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37c554-5899-46ce-aebe-ea12c2049fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:33.044734 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37c554-5899-46ce-aebe-ea12c2049fb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:33.045393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f37c554-5899-46ce-aebe-ea12c2049fb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:33.047352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f37c554-5899-46ce-aebe-ea12c2049fb7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:33.075910 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1273/2580] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:32 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:33.080288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692b127-962d-4a4f-9c5a-2ea1a31173f2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:33.091321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692b127-962d-4a4f-9c5a-2ea1a31173f2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:33.151007 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e3690-680c-4946-8615-c7a675d1c721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:33.151639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e3690-680c-4946-8615-c7a675d1c721 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:33.151922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e3690-680c-4946-8615-c7a675d1c721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:33.152195 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e3690-680c-4946-8615-c7a675d1c721 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:33.152877 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3e3690-680c-4946-8615-c7a675d1c721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:33.154902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3e3690-680c-4946-8615-c7a675d1c721 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:33.178036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1308/2581] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:33 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:33.182181 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37c554-5899-46ce-aebe-ea12c2049fb7 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:33.193681 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37c554-5899-46ce-aebe-ea12c2049fb7 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:33.199765 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290f43c-55ea-48ed-b742-5ec2023856e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:33.200402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290f43c-55ea-48ed-b742-5ec2023856e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:33.200680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290f43c-55ea-48ed-b742-5ec2023856e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:33.200952 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290f43c-55ea-48ed-b742-5ec2023856e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:33.201651 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9290f43c-55ea-48ed-b742-5ec2023856e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:33.203312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9290f43c-55ea-48ed-b742-5ec2023856e2 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:33.214537 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1274/2582] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:33.218840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e3690-680c-4946-8615-c7a675d1c721 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:33.229512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e3690-680c-4946-8615-c7a675d1c721 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:33.234402 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0709a95-e930-4504-a6ce-396668994089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:33.235076 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0709a95-e930-4504-a6ce-396668994089 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:33.235354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0709a95-e930-4504-a6ce-396668994089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:33.235629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0709a95-e930-4504-a6ce-396668994089 admin admin] PATH_INFO: `/v3/projects/5966d3851ffd41aab1d5c9ef036e068d/users/ce4b26364a7047a69f9193705e861974/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:33.252849 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0709a95-e930-4504-a6ce-396668994089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5966d3851ffd41aab1d5c9ef036e068d, user_id=ce4b26364a7047a69f9193705e861974, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:33.255533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0709a95-e930-4504-a6ce-396668994089 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:33.269738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1309/2583] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:33 2022] PUT /identity/v3/projects/5966d3851ffd41aab1d5c9ef036e068d/users/ce4b26364a7047a69f9193705e861974/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:33.273738 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290f43c-55ea-48ed-b742-5ec2023856e2 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:33.285731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290f43c-55ea-48ed-b742-5ec2023856e2 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:33.348538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915689d2-dbc8-41ca-b205-b18ee754d7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:33.349172 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915689d2-dbc8-41ca-b205-b18ee754d7fe admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:33.349458 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915689d2-dbc8-41ca-b205-b18ee754d7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:33.349741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915689d2-dbc8-41ca-b205-b18ee754d7fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:33.350478 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915689d2-dbc8-41ca-b205-b18ee754d7fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:33.352093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915689d2-dbc8-41ca-b205-b18ee754d7fe admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:33.363450 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1275/2584] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:33.367752 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0709a95-e930-4504-a6ce-396668994089 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:33.379629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0709a95-e930-4504-a6ce-396668994089 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:33.384442 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9d951-c492-438f-bd73-071284cc01fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:33.385132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9d951-c492-438f-bd73-071284cc01fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:33.385415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9d951-c492-438f-bd73-071284cc01fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:33.385688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9d951-c492-438f-bd73-071284cc01fc admin admin] PATH_INFO: `/v3/projects/5966d3851ffd41aab1d5c9ef036e068d/users/ce4b26364a7047a69f9193705e861974/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:33.387743 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9d951-c492-438f-bd73-071284cc01fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5966d3851ffd41aab1d5c9ef036e068d, user_id=ce4b26364a7047a69f9193705e861974, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:33.390430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9d951-c492-438f-bd73-071284cc01fc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:33.401685 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1310/2585] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:33 2022] PUT /identity/v3/projects/5966d3851ffd41aab1d5c9ef036e068d/users/ce4b26364a7047a69f9193705e861974/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:33.413917 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2737c-af30-4369-9c95-dde0531aafd5 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:33.414223 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2737c-af30-4369-9c95-dde0531aafd5 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:33.414500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2737c-af30-4369-9c95-dde0531aafd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:33.445308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ddf2737c-af30-4369-9c95-dde0531aafd5 None None] MFA Rules not processed for user `ce4b26364a7047a69f9193705e861974`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:33.446569 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ddf2737c-af30-4369-9c95-dde0531aafd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:33.457046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:33.520684 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1276/2586] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:33 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:33.531999 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9d951-c492-438f-bd73-071284cc01fc admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:33.534346 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a0d9d951-c492-438f-bd73-071284cc01fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:33.542747 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:33.605589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9d951-c492-438f-bd73-071284cc01fc admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:33.612149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d7066-879b-442c-9914-4839b7f22800 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:33.612149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d7066-879b-442c-9914-4839b7f22800 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:33.612149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d7066-879b-442c-9914-4839b7f22800 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:33.612149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d7066-879b-442c-9914-4839b7f22800 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:33.622356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d7066-879b-442c-9914-4839b7f22800 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:33.623831 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d7066-879b-442c-9914-4839b7f22800 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:33.634817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1311/2587] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:34.065110 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf2737c-af30-4369-9c95-dde0531aafd5 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:34.129164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf2737c-af30-4369-9c95-dde0531aafd5 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:34.134429 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320cb85-51ed-40db-a08c-3fa307ad465b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:34.135081 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320cb85-51ed-40db-a08c-3fa307ad465b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:34.135368 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320cb85-51ed-40db-a08c-3fa307ad465b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:34.135652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320cb85-51ed-40db-a08c-3fa307ad465b admin admin] PATH_INFO: `/v3/users/df1fa22d85c349adbd72bb0cfcb31bd9` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:34.136826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320cb85-51ed-40db-a08c-3fa307ad465b admin admin] RBAC: Authorizing `identity:delete_user(user_id=df1fa22d85c349adbd72bb0cfcb31bd9)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:34.138885 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320cb85-51ed-40db-a08c-3fa307ad465b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:34.193107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5320cb85-51ed-40db-a08c-3fa307ad465b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df1fa22d85c349adbd72bb0cfcb31bd9'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:34.210217 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5320cb85-51ed-40db-a08c-3fa307ad465b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df1fa22d85c349adbd72bb0cfcb31bd9'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:34.210217 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5320cb85-51ed-40db-a08c-3fa307ad465b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df1fa22d85c349adbd72bb0cfcb31bd9'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:34.221052 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5320cb85-51ed-40db-a08c-3fa307ad465b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df1fa22d85c349adbd72bb0cfcb31bd9'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:34.221710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5320cb85-51ed-40db-a08c-3fa307ad465b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df1fa22d85c349adbd72bb0cfcb31bd9'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:34.234514 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1277/2588] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:34 2022] DELETE /identity/v3/users/df1fa22d85c349adbd72bb0cfcb31bd9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:34.239577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22886310-373e-47cf-8e94-a2d57e34316d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:34.239872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22886310-373e-47cf-8e94-a2d57e34316d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:34.240164 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22886310-373e-47cf-8e94-a2d57e34316d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:34.259553 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-22886310-373e-47cf-8e94-a2d57e34316d None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:34.260773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-22886310-373e-47cf-8e94-a2d57e34316d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:34.326911 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1312/2589] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:34.337981 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320cb85-51ed-40db-a08c-3fa307ad465b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:34.339566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5320cb85-51ed-40db-a08c-3fa307ad465b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:34.371826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:34.400482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320cb85-51ed-40db-a08c-3fa307ad465b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:34.405026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5773771-7002-41e0-a5e8-163b3e71daec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:34.406066 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5773771-7002-41e0-a5e8-163b3e71daec service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:34.406401 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5773771-7002-41e0-a5e8-163b3e71daec service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:34.406794 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5773771-7002-41e0-a5e8-163b3e71daec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:34.418810 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5773771-7002-41e0-a5e8-163b3e71daec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:34.420267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5773771-7002-41e0-a5e8-163b3e71daec service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:34.431375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1278/2590] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:34.722905 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22886310-373e-47cf-8e94-a2d57e34316d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:34.724465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-22886310-373e-47cf-8e94-a2d57e34316d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:34.739749 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22886310-373e-47cf-8e94-a2d57e34316d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:34.744867 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad92f4-2896-4605-a9ed-7d42e74e1371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:34.745599 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad92f4-2896-4605-a9ed-7d42e74e1371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:34.745968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad92f4-2896-4605-a9ed-7d42e74e1371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:34.746195 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad92f4-2896-4605-a9ed-7d42e74e1371 admin admin] PATH_INFO: `/v3/projects/057c1d6ce34c42d3af74548086a3e64c` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:34.747462 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad92f4-2896-4605-a9ed-7d42e74e1371 admin admin] RBAC: Authorizing `identity:delete_project(project_id=057c1d6ce34c42d3af74548086a3e64c)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:34.749568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad92f4-2896-4605-a9ed-7d42e74e1371 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:34.805166 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-41ad92f4-2896-4605-a9ed-7d42e74e1371 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '057c1d6ce34c42d3af74548086a3e64c', 'request_id': 'req-41ad92f4-2896-4605-a9ed-7d42e74e1371'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:34.811742 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-41ad92f4-2896-4605-a9ed-7d42e74e1371 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '057c1d6ce34c42d3af74548086a3e64c', 'request_id': 'req-41ad92f4-2896-4605-a9ed-7d42e74e1371'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:34.818066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-41ad92f4-2896-4605-a9ed-7d42e74e1371 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '057c1d6ce34c42d3af74548086a3e64c', 'request_id': 'req-41ad92f4-2896-4605-a9ed-7d42e74e1371'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:34.833835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-41ad92f4-2896-4605-a9ed-7d42e74e1371 admin admin] The token cache is being invalidate because project 057c1d6ce34c42d3af74548086a3e64c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:34.834159 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-41ad92f4-2896-4605-a9ed-7d42e74e1371 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:34.834937 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-41ad92f4-2896-4605-a9ed-7d42e74e1371 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:34.836591 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1313/2591] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:34 2022] DELETE /identity/v3/projects/057c1d6ce34c42d3af74548086a3e64c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:34.840988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5773771-7002-41e0-a5e8-163b3e71daec service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:34.852274 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5773771-7002-41e0-a5e8-163b3e71daec service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:34.919440 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fa7eb-e103-4e9d-8eb4-34f175999e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:34.920061 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fa7eb-e103-4e9d-8eb4-34f175999e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:34.920346 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fa7eb-e103-4e9d-8eb4-34f175999e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:34.920625 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fa7eb-e103-4e9d-8eb4-34f175999e73 admin admin] PATH_INFO: `/v3/users/ce4b26364a7047a69f9193705e861974` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:34.921806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fa7eb-e103-4e9d-8eb4-34f175999e73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce4b26364a7047a69f9193705e861974)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:34.924531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fa7eb-e103-4e9d-8eb4-34f175999e73 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:34.979005 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-434fa7eb-e103-4e9d-8eb4-34f175999e73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce4b26364a7047a69f9193705e861974'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:34.979909 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-434fa7eb-e103-4e9d-8eb4-34f175999e73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce4b26364a7047a69f9193705e861974'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:34.992532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-434fa7eb-e103-4e9d-8eb4-34f175999e73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce4b26364a7047a69f9193705e861974'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:35.014641 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-434fa7eb-e103-4e9d-8eb4-34f175999e73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce4b26364a7047a69f9193705e861974'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:35.015487 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-434fa7eb-e103-4e9d-8eb4-34f175999e73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce4b26364a7047a69f9193705e861974'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:35.028856 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1279/2592] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:34 2022] DELETE /identity/v3/users/ce4b26364a7047a69f9193705e861974 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:35.342416 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad92f4-2896-4605-a9ed-7d42e74e1371 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:35.344474 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-41ad92f4-2896-4605-a9ed-7d42e74e1371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:35.366106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad92f4-2896-4605-a9ed-7d42e74e1371 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:35.398323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:35.431487 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a3526-3cf3-41b8-a191-8087d3a24557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:35.432075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a3526-3cf3-41b8-a191-8087d3a24557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:35.432365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a3526-3cf3-41b8-a191-8087d3a24557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:35.432688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a3526-3cf3-41b8-a191-8087d3a24557 admin admin] PATH_INFO: `/v3/projects/5966d3851ffd41aab1d5c9ef036e068d` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:35.433831 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a3526-3cf3-41b8-a191-8087d3a24557 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5966d3851ffd41aab1d5c9ef036e068d)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:35.435808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a3526-3cf3-41b8-a191-8087d3a24557 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:35.487113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d78a3526-3cf3-41b8-a191-8087d3a24557 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5966d3851ffd41aab1d5c9ef036e068d', 'request_id': 'req-d78a3526-3cf3-41b8-a191-8087d3a24557'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:35.492983 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d78a3526-3cf3-41b8-a191-8087d3a24557 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5966d3851ffd41aab1d5c9ef036e068d', 'request_id': 'req-d78a3526-3cf3-41b8-a191-8087d3a24557'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:35.499468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d78a3526-3cf3-41b8-a191-8087d3a24557 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5966d3851ffd41aab1d5c9ef036e068d', 'request_id': 'req-d78a3526-3cf3-41b8-a191-8087d3a24557'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:35.513034 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d78a3526-3cf3-41b8-a191-8087d3a24557 admin admin] The token cache is being invalidate because project 5966d3851ffd41aab1d5c9ef036e068d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:35.513323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d78a3526-3cf3-41b8-a191-8087d3a24557 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:35.514071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d78a3526-3cf3-41b8-a191-8087d3a24557 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:35.515654 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1314/2593] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:35 2022] DELETE /identity/v3/projects/5966d3851ffd41aab1d5c9ef036e068d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:35.521342 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb7966-6b7d-4493-9982-99b38e679e39 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:35.521879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb7966-6b7d-4493-9982-99b38e679e39 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:35.522025 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb7966-6b7d-4493-9982-99b38e679e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:35.544840 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-21bb7966-6b7d-4493-9982-99b38e679e39 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:35.546389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-21bb7966-6b7d-4493-9982-99b38e679e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:35.594247 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:35.611095 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1280/2594] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:35.620121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d8fe6-e9bc-4ba3-96ec-0aa3311ba7d8 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:35.620405 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d8fe6-e9bc-4ba3-96ec-0aa3311ba7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:35.621376 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d8fe6-e9bc-4ba3-96ec-0aa3311ba7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:35.641502 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8c7d8fe6-e9bc-4ba3-96ec-0aa3311ba7d8 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:35.643407 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8c7d8fe6-e9bc-4ba3-96ec-0aa3311ba7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:35.651348 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1315/2595] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:35.655904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb7966-6b7d-4493-9982-99b38e679e39 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:35.666141 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb7966-6b7d-4493-9982-99b38e679e39 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:35.670990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb254881-6eff-4425-aad7-86074482d881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:35.671526 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb254881-6eff-4425-aad7-86074482d881 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:35.671828 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb254881-6eff-4425-aad7-86074482d881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:35.672131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb254881-6eff-4425-aad7-86074482d881 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:35.672803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb254881-6eff-4425-aad7-86074482d881 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:09:35.673113 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb254881-6eff-4425-aad7-86074482d881 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:35.674805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb254881-6eff-4425-aad7-86074482d881 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:35.687357 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1281/2596] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:09:35.692280 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d8fe6-e9bc-4ba3-96ec-0aa3311ba7d8 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:35.702082 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d8fe6-e9bc-4ba3-96ec-0aa3311ba7d8 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:35.707463 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382064ad-f7c9-4668-b6fd-4deda4e0aabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:35.708092 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382064ad-f7c9-4668-b6fd-4deda4e0aabb admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:35.708373 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382064ad-f7c9-4668-b6fd-4deda4e0aabb admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:35.708645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382064ad-f7c9-4668-b6fd-4deda4e0aabb admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:35.709311 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382064ad-f7c9-4668-b6fd-4deda4e0aabb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:35.711262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382064ad-f7c9-4668-b6fd-4deda4e0aabb admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:35.740477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1316/2597] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:35 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:35.745514 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb254881-6eff-4425-aad7-86074482d881 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:35.756301 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb254881-6eff-4425-aad7-86074482d881 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:35.817418 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aed534-c504-408f-a9c2-3208ee05a981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:35.818087 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aed534-c504-408f-a9c2-3208ee05a981 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:35.818377 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aed534-c504-408f-a9c2-3208ee05a981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:35.818658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aed534-c504-408f-a9c2-3208ee05a981 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:35.819346 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aed534-c504-408f-a9c2-3208ee05a981 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:35.821310 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aed534-c504-408f-a9c2-3208ee05a981 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:35.847887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1282/2598] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:35 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:35.851310 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382064ad-f7c9-4668-b6fd-4deda4e0aabb admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:35.862418 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382064ad-f7c9-4668-b6fd-4deda4e0aabb admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:35.867345 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2dd68-8159-4911-8a02-7771a05b6f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:35.867968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2dd68-8159-4911-8a02-7771a05b6f72 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:35.868247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2dd68-8159-4911-8a02-7771a05b6f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:35.868517 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2dd68-8159-4911-8a02-7771a05b6f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:35.869207 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2dd68-8159-4911-8a02-7771a05b6f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:35.870833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2dd68-8159-4911-8a02-7771a05b6f72 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:35.882273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1317/2599] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:35.887211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aed534-c504-408f-a9c2-3208ee05a981 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:35.898250 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aed534-c504-408f-a9c2-3208ee05a981 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:35.904721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cd5d1-a2c8-4f84-ad2f-1b7e990abe68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:35.905592 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cd5d1-a2c8-4f84-ad2f-1b7e990abe68 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:35.905878 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cd5d1-a2c8-4f84-ad2f-1b7e990abe68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:35.906186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cd5d1-a2c8-4f84-ad2f-1b7e990abe68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:35.908170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cd5d1-a2c8-4f84-ad2f-1b7e990abe68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:35.908620 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cd5d1-a2c8-4f84-ad2f-1b7e990abe68 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:35.920598 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1283/2600] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:35.925505 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2dd68-8159-4911-8a02-7771a05b6f72 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:35.935991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2dd68-8159-4911-8a02-7771a05b6f72 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:35.941597 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba52c44-4979-4650-a428-ecde09a0756c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:35.942312 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba52c44-4979-4650-a428-ecde09a0756c admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:35.942603 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba52c44-4979-4650-a428-ecde09a0756c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:35.942879 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba52c44-4979-4650-a428-ecde09a0756c admin admin] PATH_INFO: `/v3/projects/9410b1a201fd41b485489a8bff5cbfa3/users/c7d7122be44442e4bdae4511e9ff7ec0/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:35.958736 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba52c44-4979-4650-a428-ecde09a0756c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9410b1a201fd41b485489a8bff5cbfa3, user_id=c7d7122be44442e4bdae4511e9ff7ec0, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:35.961401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba52c44-4979-4650-a428-ecde09a0756c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:35.975051 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1318/2601] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:35 2022] PUT /identity/v3/projects/9410b1a201fd41b485489a8bff5cbfa3/users/c7d7122be44442e4bdae4511e9ff7ec0/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:35.987449 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce74c3-6be4-4e80-90c0-4391cf9aeb6d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:35.988082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce74c3-6be4-4e80-90c0-4391cf9aeb6d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:35.988266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce74c3-6be4-4e80-90c0-4391cf9aeb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:36.019605 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-51ce74c3-6be4-4e80-90c0-4391cf9aeb6d None None] MFA Rules not processed for user `c7d7122be44442e4bdae4511e9ff7ec0`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:36.020933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-51ce74c3-6be4-4e80-90c0-4391cf9aeb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:36.094326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1284/2602] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:09:36.099312 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba52c44-4979-4650-a428-ecde09a0756c admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:36.161653 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:36.165688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba52c44-4979-4650-a428-ecde09a0756c admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:36.172821 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777607f-34bf-43c0-9b22-f17456d69c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:36.174019 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777607f-34bf-43c0-9b22-f17456d69c43 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:36.174481 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777607f-34bf-43c0-9b22-f17456d69c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:36.174933 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777607f-34bf-43c0-9b22-f17456d69c43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:36.175963 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9777607f-34bf-43c0-9b22-f17456d69c43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:36.178027 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9777607f-34bf-43c0-9b22-f17456d69c43 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:36.212366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1319/2603] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:36 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:36.216654 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce74c3-6be4-4e80-90c0-4391cf9aeb6d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:36.229320 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce74c3-6be4-4e80-90c0-4391cf9aeb6d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:36.275914 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:36.308699 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7727f-8881-4e45-bbf2-2d6c6b7992e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:36.309793 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7727f-8881-4e45-bbf2-2d6c6b7992e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:36.310669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7727f-8881-4e45-bbf2-2d6c6b7992e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:36.311115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7727f-8881-4e45-bbf2-2d6c6b7992e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:36.312035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a7727f-8881-4e45-bbf2-2d6c6b7992e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:36.315689 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a7727f-8881-4e45-bbf2-2d6c6b7992e1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:36.344598 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1285/2604] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:36 2022] POST /identity/v3/users => generated 299 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:36.349390 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777607f-34bf-43c0-9b22-f17456d69c43 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:36.361943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777607f-34bf-43c0-9b22-f17456d69c43 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:36.367382 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a80ab-b2be-42c3-a362-b945063d9d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:36.368015 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a80ab-b2be-42c3-a362-b945063d9d03 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:36.368297 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a80ab-b2be-42c3-a362-b945063d9d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:36.368566 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a80ab-b2be-42c3-a362-b945063d9d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:36.369294 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a80ab-b2be-42c3-a362-b945063d9d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:36.371218 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a80ab-b2be-42c3-a362-b945063d9d03 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:36.389319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1320/2605] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:36.395657 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7727f-8881-4e45-bbf2-2d6c6b7992e1 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:36.406041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7727f-8881-4e45-bbf2-2d6c6b7992e1 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:36.412696 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb141a-c3de-4567-9a97-309207fc49b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:36.413866 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb141a-c3de-4567-9a97-309207fc49b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:36.414398 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb141a-c3de-4567-9a97-309207fc49b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:36.414986 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb141a-c3de-4567-9a97-309207fc49b9 admin admin] PATH_INFO: `/v3/projects/163d33c680644f08bbba06fe1b7d7365/users/0741bd92dabb4cbb84ea8a22d8013285/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:36.434623 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb141a-c3de-4567-9a97-309207fc49b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=163d33c680644f08bbba06fe1b7d7365, user_id=0741bd92dabb4cbb84ea8a22d8013285, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:36.437320 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb141a-c3de-4567-9a97-309207fc49b9 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:36.461031 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1286/2606] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:36 2022] PUT /identity/v3/projects/163d33c680644f08bbba06fe1b7d7365/users/0741bd92dabb4cbb84ea8a22d8013285/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:36.463256 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a80ab-b2be-42c3-a362-b945063d9d03 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:36.476173 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a80ab-b2be-42c3-a362-b945063d9d03 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:36.546049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa0026-2186-4460-aba3-85ca420c59a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:36.546684 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa0026-2186-4460-aba3-85ca420c59a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:36.546969 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa0026-2186-4460-aba3-85ca420c59a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:36.547249 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa0026-2186-4460-aba3-85ca420c59a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:36.547951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa0026-2186-4460-aba3-85ca420c59a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:36.550118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa0026-2186-4460-aba3-85ca420c59a2 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:36.560534 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1321/2607] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:36.564795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb141a-c3de-4567-9a97-309207fc49b9 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:36.575620 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb141a-c3de-4567-9a97-309207fc49b9 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:36.580531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f63cc3-20d1-4ef2-ba23-5e79e52dde88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:36.581224 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f63cc3-20d1-4ef2-ba23-5e79e52dde88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:36.581508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f63cc3-20d1-4ef2-ba23-5e79e52dde88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:36.581794 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f63cc3-20d1-4ef2-ba23-5e79e52dde88 admin admin] PATH_INFO: `/v3/projects/163d33c680644f08bbba06fe1b7d7365/users/0741bd92dabb4cbb84ea8a22d8013285/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:36.584309 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f63cc3-20d1-4ef2-ba23-5e79e52dde88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=163d33c680644f08bbba06fe1b7d7365, user_id=0741bd92dabb4cbb84ea8a22d8013285, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:36.587141 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f63cc3-20d1-4ef2-ba23-5e79e52dde88 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:36.599064 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1287/2608] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:36 2022] PUT /identity/v3/projects/163d33c680644f08bbba06fe1b7d7365/users/0741bd92dabb4cbb84ea8a22d8013285/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:36.609943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4659e-7f1f-4ed0-8a8f-bd88325b7c17 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:36.610234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4659e-7f1f-4ed0-8a8f-bd88325b7c17 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:36.610512 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4659e-7f1f-4ed0-8a8f-bd88325b7c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:36.641520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-48d4659e-7f1f-4ed0-8a8f-bd88325b7c17 None None] MFA Rules not processed for user `0741bd92dabb4cbb84ea8a22d8013285`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:36.643585 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-48d4659e-7f1f-4ed0-8a8f-bd88325b7c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:36.720922 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1322/2609] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:36.733465 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f63cc3-20d1-4ef2-ba23-5e79e52dde88 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:36.735337 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-03f63cc3-20d1-4ef2-ba23-5e79e52dde88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:36.800129 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f63cc3-20d1-4ef2-ba23-5e79e52dde88 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:36.803847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a332f-451f-479c-b93d-05c6833090f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:36.804608 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a332f-451f-479c-b93d-05c6833090f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:36.804892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a332f-451f-479c-b93d-05c6833090f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:36.805166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a332f-451f-479c-b93d-05c6833090f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:36.868941 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a332f-451f-479c-b93d-05c6833090f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:36.870449 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a332f-451f-479c-b93d-05c6833090f6 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:36.879760 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1288/2610] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:40.579885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4659e-7f1f-4ed0-8a8f-bd88325b7c17 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:40.581348 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-48d4659e-7f1f-4ed0-8a8f-bd88325b7c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:40.649331 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:40.665466 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4659e-7f1f-4ed0-8a8f-bd88325b7c17 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:40.672299 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4432c4-bcb2-4a68-aa29-2a89e3a47b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:40.673068 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4432c4-bcb2-4a68-aa29-2a89e3a47b72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:40.673952 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4432c4-bcb2-4a68-aa29-2a89e3a47b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:40.674286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4432c4-bcb2-4a68-aa29-2a89e3a47b72 admin admin] PATH_INFO: `/v3/users/e23c38fff394405c99b924d247d0d70b` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:40.675465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4432c4-bcb2-4a68-aa29-2a89e3a47b72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e23c38fff394405c99b924d247d0d70b)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:40.677436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4432c4-bcb2-4a68-aa29-2a89e3a47b72 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:40.751269 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8f4432c4-bcb2-4a68-aa29-2a89e3a47b72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e23c38fff394405c99b924d247d0d70b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:40.752122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8f4432c4-bcb2-4a68-aa29-2a89e3a47b72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e23c38fff394405c99b924d247d0d70b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:40.763387 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8f4432c4-bcb2-4a68-aa29-2a89e3a47b72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e23c38fff394405c99b924d247d0d70b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:40.789573 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8f4432c4-bcb2-4a68-aa29-2a89e3a47b72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e23c38fff394405c99b924d247d0d70b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:40.801482 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8f4432c4-bcb2-4a68-aa29-2a89e3a47b72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e23c38fff394405c99b924d247d0d70b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:40.810697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1323/2611] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:40 2022] DELETE /identity/v3/users/e23c38fff394405c99b924d247d0d70b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:40.823688 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612baf7-2d89-44e6-9365-42e7b6a38c89 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:40.824748 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612baf7-2d89-44e6-9365-42e7b6a38c89 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:40.825294 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612baf7-2d89-44e6-9365-42e7b6a38c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:40.855002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-1612baf7-2d89-44e6-9365-42e7b6a38c89 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:40.857314 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1612baf7-2d89-44e6-9365-42e7b6a38c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:40.918346 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:40.934116 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1289/2612] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:40.945267 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4432c4-bcb2-4a68-aa29-2a89e3a47b72 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:40.948321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8f4432c4-bcb2-4a68-aa29-2a89e3a47b72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:41.017633 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4432c4-bcb2-4a68-aa29-2a89e3a47b72 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:41.021664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8211d639-c79a-494d-b704-361013b37aed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:41.022335 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211d639-c79a-494d-b704-361013b37aed service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:41.022770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211d639-c79a-494d-b704-361013b37aed service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:41.022906 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211d639-c79a-494d-b704-361013b37aed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:41.033493 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8211d639-c79a-494d-b704-361013b37aed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:41.035016 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8211d639-c79a-494d-b704-361013b37aed service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:41.045247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1324/2613] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:41.340143 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612baf7-2d89-44e6-9365-42e7b6a38c89 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:41.342204 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1612baf7-2d89-44e6-9365-42e7b6a38c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:41.357642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612baf7-2d89-44e6-9365-42e7b6a38c89 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:41.364206 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180f223-71d5-4786-8801-b1123265aa86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:41.365494 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180f223-71d5-4786-8801-b1123265aa86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:41.366226 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180f223-71d5-4786-8801-b1123265aa86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:41.366529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180f223-71d5-4786-8801-b1123265aa86 admin admin] PATH_INFO: `/v3/projects/cb564e3e495b49eea4b0e2477e812577` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:41.368029 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180f223-71d5-4786-8801-b1123265aa86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb564e3e495b49eea4b0e2477e812577)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:41.369904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180f223-71d5-4786-8801-b1123265aa86 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:41.425846 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e180f223-71d5-4786-8801-b1123265aa86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb564e3e495b49eea4b0e2477e812577', 'request_id': 'req-e180f223-71d5-4786-8801-b1123265aa86'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:41.433350 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e180f223-71d5-4786-8801-b1123265aa86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb564e3e495b49eea4b0e2477e812577', 'request_id': 'req-e180f223-71d5-4786-8801-b1123265aa86'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:41.439953 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e180f223-71d5-4786-8801-b1123265aa86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb564e3e495b49eea4b0e2477e812577', 'request_id': 'req-e180f223-71d5-4786-8801-b1123265aa86'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:41.453414 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e180f223-71d5-4786-8801-b1123265aa86 admin admin] The token cache is being invalidate because project cb564e3e495b49eea4b0e2477e812577 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:41.453638 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e180f223-71d5-4786-8801-b1123265aa86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:41.454266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e180f223-71d5-4786-8801-b1123265aa86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:41.456573 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1290/2614] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:41 2022] DELETE /identity/v3/projects/cb564e3e495b49eea4b0e2477e812577 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:41.461952 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4ff31-2b53-4e5a-8fa9-c5c1fd978611 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:41.462244 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4ff31-2b53-4e5a-8fa9-c5c1fd978611 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:41.462522 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4ff31-2b53-4e5a-8fa9-c5c1fd978611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:41.483610 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-0ae4ff31-2b53-4e5a-8fa9-c5c1fd978611 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:41.484857 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0ae4ff31-2b53-4e5a-8fa9-c5c1fd978611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:41.547248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1325/2615] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:41.556286 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3902bba-b87d-4610-b2b2-3235911e5896 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:41.556574 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3902bba-b87d-4610-b2b2-3235911e5896 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:41.556850 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3902bba-b87d-4610-b2b2-3235911e5896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:41.578288 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e3902bba-b87d-4610-b2b2-3235911e5896 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:41.579517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e3902bba-b87d-4610-b2b2-3235911e5896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:41.592037 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1291/2616] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:41.596430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4ff31-2b53-4e5a-8fa9-c5c1fd978611 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:41.608234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4ff31-2b53-4e5a-8fa9-c5c1fd978611 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:41.614079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c019c6-2e71-44e0-bf22-17c3606e8a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:41.614712 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c019c6-2e71-44e0-bf22-17c3606e8a50 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:41.614996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c019c6-2e71-44e0-bf22-17c3606e8a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:41.615268 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c019c6-2e71-44e0-bf22-17c3606e8a50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:41.615975 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c019c6-2e71-44e0-bf22-17c3606e8a50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:09:41.616280 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c019c6-2e71-44e0-bf22-17c3606e8a50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:41.620317 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c019c6-2e71-44e0-bf22-17c3606e8a50 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:41.631740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1326/2617] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:09:41.638201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3902bba-b87d-4610-b2b2-3235911e5896 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:41.648457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3902bba-b87d-4610-b2b2-3235911e5896 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:41.653615 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361c4ac-cf40-4836-a407-86fa0e26f7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:41.654286 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361c4ac-cf40-4836-a407-86fa0e26f7da admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:41.654576 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361c4ac-cf40-4836-a407-86fa0e26f7da admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:41.654852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361c4ac-cf40-4836-a407-86fa0e26f7da admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:41.655658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361c4ac-cf40-4836-a407-86fa0e26f7da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:41.657577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361c4ac-cf40-4836-a407-86fa0e26f7da admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:41.687312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1292/2618] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:41 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:41.692377 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c019c6-2e71-44e0-bf22-17c3606e8a50 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:41.698115 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:41.707519 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c019c6-2e71-44e0-bf22-17c3606e8a50 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:41.769657 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c8dc6-6265-44a2-80fb-d97abedb765b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:41.770358 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c8dc6-6265-44a2-80fb-d97abedb765b admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:41.770635 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c8dc6-6265-44a2-80fb-d97abedb765b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:41.770903 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c8dc6-6265-44a2-80fb-d97abedb765b admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:41.771566 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c8dc6-6265-44a2-80fb-d97abedb765b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:41.773472 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c8dc6-6265-44a2-80fb-d97abedb765b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:41.795172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1327/2619] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:41 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:41.799952 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361c4ac-cf40-4836-a407-86fa0e26f7da admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:41.812321 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361c4ac-cf40-4836-a407-86fa0e26f7da admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:41.817554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065efa3-37b4-4e0d-847b-b6c0c2947e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:41.818225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065efa3-37b4-4e0d-847b-b6c0c2947e3e admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:41.818509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065efa3-37b4-4e0d-847b-b6c0c2947e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:41.818788 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065efa3-37b4-4e0d-847b-b6c0c2947e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:41.819483 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065efa3-37b4-4e0d-847b-b6c0c2947e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:41.821095 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065efa3-37b4-4e0d-847b-b6c0c2947e3e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:41.833427 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1293/2620] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:41.838899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c8dc6-6265-44a2-80fb-d97abedb765b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:41.850411 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c8dc6-6265-44a2-80fb-d97abedb765b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:41.857746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d854b-fc0a-4271-a7e8-e4069cf52263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:41.859138 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d854b-fc0a-4271-a7e8-e4069cf52263 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:41.859138 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d854b-fc0a-4271-a7e8-e4069cf52263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:41.859138 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d854b-fc0a-4271-a7e8-e4069cf52263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:41.859657 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d854b-fc0a-4271-a7e8-e4069cf52263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:41.861876 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d854b-fc0a-4271-a7e8-e4069cf52263 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:41.881486 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1328/2621] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:41.882781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065efa3-37b4-4e0d-847b-b6c0c2947e3e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:41.895636 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065efa3-37b4-4e0d-847b-b6c0c2947e3e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:41.902813 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e86d7-ca9b-453d-a755-03974f4eb12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:41.903728 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e86d7-ca9b-453d-a755-03974f4eb12c admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:41.904239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e86d7-ca9b-453d-a755-03974f4eb12c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:41.904239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e86d7-ca9b-453d-a755-03974f4eb12c admin admin] PATH_INFO: `/v3/projects/4ac863e436fc44199b9e6d7d952b973f/users/bd2b4005960b4bfb9ab7c37507e15ee0/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:41.930616 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e86d7-ca9b-453d-a755-03974f4eb12c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ac863e436fc44199b9e6d7d952b973f, user_id=bd2b4005960b4bfb9ab7c37507e15ee0, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:41.933542 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e86d7-ca9b-453d-a755-03974f4eb12c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:41.958755 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1294/2622] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:41 2022] PUT /identity/v3/projects/4ac863e436fc44199b9e6d7d952b973f/users/bd2b4005960b4bfb9ab7c37507e15ee0/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:41.972393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c761866-b14d-44a7-9ea0-9370e091b5d5 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:41.972393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c761866-b14d-44a7-9ea0-9370e091b5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:41.972393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c761866-b14d-44a7-9ea0-9370e091b5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:42.005580 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-5c761866-b14d-44a7-9ea0-9370e091b5d5 None None] MFA Rules not processed for user `bd2b4005960b4bfb9ab7c37507e15ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:42.007084 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5c761866-b14d-44a7-9ea0-9370e091b5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:42.088032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1329/2623] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:42.099604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e86d7-ca9b-453d-a755-03974f4eb12c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:42.101336 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bf6e86d7-ca9b-453d-a755-03974f4eb12c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:42.172248 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e86d7-ca9b-453d-a755-03974f4eb12c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:42.177746 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793af47d-dbfe-47e2-96e6-00ff4aa1fb78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:42.178432 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793af47d-dbfe-47e2-96e6-00ff4aa1fb78 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:42.178724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793af47d-dbfe-47e2-96e6-00ff4aa1fb78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:42.179004 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793af47d-dbfe-47e2-96e6-00ff4aa1fb78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:42.190180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793af47d-dbfe-47e2-96e6-00ff4aa1fb78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:42.190685 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793af47d-dbfe-47e2-96e6-00ff4aa1fb78 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:42.204258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1295/2624] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:43.506360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c761866-b14d-44a7-9ea0-9370e091b5d5 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:43.507879 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5c761866-b14d-44a7-9ea0-9370e091b5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:43.520190 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c761866-b14d-44a7-9ea0-9370e091b5d5 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:43.523779 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9e6d9-a74a-4d8e-9e28-16e6c8c46adb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:43.524399 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9e6d9-a74a-4d8e-9e28-16e6c8c46adb service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:43.524699 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9e6d9-a74a-4d8e-9e28-16e6c8c46adb service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:43.524953 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9e6d9-a74a-4d8e-9e28-16e6c8c46adb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:43.526781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9ab9e6d9-a74a-4d8e-9e28-16e6c8c46adb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:43.596819 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab9e6d9-a74a-4d8e-9e28-16e6c8c46adb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:43.598312 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab9e6d9-a74a-4d8e-9e28-16e6c8c46adb service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:43.608639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1330/2625] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:48.285480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793af47d-dbfe-47e2-96e6-00ff4aa1fb78 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:48.287201 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-793af47d-dbfe-47e2-96e6-00ff4aa1fb78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:48.305924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:48.405506 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793af47d-dbfe-47e2-96e6-00ff4aa1fb78 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:48.414686 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0226d9b-be24-42cf-817c-9a3aa57a5799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:48.415983 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0226d9b-be24-42cf-817c-9a3aa57a5799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:48.416444 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0226d9b-be24-42cf-817c-9a3aa57a5799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:48.416896 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0226d9b-be24-42cf-817c-9a3aa57a5799 admin admin] PATH_INFO: `/v3/users/0d004ec91e72491fac61653b571f18a8` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:48.418884 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0226d9b-be24-42cf-817c-9a3aa57a5799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d004ec91e72491fac61653b571f18a8)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:48.422862 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0226d9b-be24-42cf-817c-9a3aa57a5799 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:48.495120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f0226d9b-be24-42cf-817c-9a3aa57a5799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d004ec91e72491fac61653b571f18a8'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:48.496096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f0226d9b-be24-42cf-817c-9a3aa57a5799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d004ec91e72491fac61653b571f18a8'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:48.504761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f0226d9b-be24-42cf-817c-9a3aa57a5799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d004ec91e72491fac61653b571f18a8'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:48.519007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f0226d9b-be24-42cf-817c-9a3aa57a5799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d004ec91e72491fac61653b571f18a8'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:48.519817 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f0226d9b-be24-42cf-817c-9a3aa57a5799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d004ec91e72491fac61653b571f18a8'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:48.539135 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1296/2626] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:48 2022] DELETE /identity/v3/users/0d004ec91e72491fac61653b571f18a8 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:48.544912 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433db78-ee23-4dbb-967d-3ae79d258f5b None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:48.544912 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433db78-ee23-4dbb-967d-3ae79d258f5b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:48.545594 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433db78-ee23-4dbb-967d-3ae79d258f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:48.557862 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:48.573130 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1433db78-ee23-4dbb-967d-3ae79d258f5b None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:48.574665 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1433db78-ee23-4dbb-967d-3ae79d258f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:48.677204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1331/2627] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:48.688502 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0226d9b-be24-42cf-817c-9a3aa57a5799 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:48.690602 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f0226d9b-be24-42cf-817c-9a3aa57a5799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:48.764409 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0226d9b-be24-42cf-817c-9a3aa57a5799 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:48.768694 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b5e55-0f9c-4b16-90c9-005e0bdd6dad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:48.774369 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b5e55-0f9c-4b16-90c9-005e0bdd6dad service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:48.774369 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b5e55-0f9c-4b16-90c9-005e0bdd6dad service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:48.774369 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b5e55-0f9c-4b16-90c9-005e0bdd6dad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:48.782083 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b5e55-0f9c-4b16-90c9-005e0bdd6dad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:48.789070 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b5e55-0f9c-4b16-90c9-005e0bdd6dad service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:48.797195 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1297/2628] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:49.109521 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433db78-ee23-4dbb-967d-3ae79d258f5b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:49.111027 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1433db78-ee23-4dbb-967d-3ae79d258f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:49.124812 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433db78-ee23-4dbb-967d-3ae79d258f5b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:49.131994 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b9c4c-eebc-45c2-b2d5-9156b8193d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:49.132741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b9c4c-eebc-45c2-b2d5-9156b8193d85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:49.133097 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b9c4c-eebc-45c2-b2d5-9156b8193d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:49.133376 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b9c4c-eebc-45c2-b2d5-9156b8193d85 admin admin] PATH_INFO: `/v3/projects/daa1ccae25954cbcb122c5fe10baefe0` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:49.134620 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b9c4c-eebc-45c2-b2d5-9156b8193d85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daa1ccae25954cbcb122c5fe10baefe0)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:49.136788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b9c4c-eebc-45c2-b2d5-9156b8193d85 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:49.187302 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-432b9c4c-eebc-45c2-b2d5-9156b8193d85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daa1ccae25954cbcb122c5fe10baefe0', 'request_id': 'req-432b9c4c-eebc-45c2-b2d5-9156b8193d85'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:49.193864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-432b9c4c-eebc-45c2-b2d5-9156b8193d85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daa1ccae25954cbcb122c5fe10baefe0', 'request_id': 'req-432b9c4c-eebc-45c2-b2d5-9156b8193d85'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:49.200503 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-432b9c4c-eebc-45c2-b2d5-9156b8193d85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daa1ccae25954cbcb122c5fe10baefe0', 'request_id': 'req-432b9c4c-eebc-45c2-b2d5-9156b8193d85'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:49.215029 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-432b9c4c-eebc-45c2-b2d5-9156b8193d85 admin admin] The token cache is being invalidate because project daa1ccae25954cbcb122c5fe10baefe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:49.215289 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-432b9c4c-eebc-45c2-b2d5-9156b8193d85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:49.215930 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-432b9c4c-eebc-45c2-b2d5-9156b8193d85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:49.217628 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1332/2629] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:49 2022] DELETE /identity/v3/projects/daa1ccae25954cbcb122c5fe10baefe0 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:49.222284 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b5e55-0f9c-4b16-90c9-005e0bdd6dad service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:49.233689 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b5e55-0f9c-4b16-90c9-005e0bdd6dad service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:49.270380 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:49.297068 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba6c93-65e4-46fc-a612-788ceded42e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:49.297805 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba6c93-65e4-46fc-a612-788ceded42e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:49.298139 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba6c93-65e4-46fc-a612-788ceded42e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:49.298309 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba6c93-65e4-46fc-a612-788ceded42e4 admin admin] PATH_INFO: `/v3/users/4888f55c998f4cb7b027f12e54b75cfa` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:49.299514 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba6c93-65e4-46fc-a612-788ceded42e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4888f55c998f4cb7b027f12e54b75cfa)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:49.301451 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba6c93-65e4-46fc-a612-788ceded42e4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:49.350129 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-55ba6c93-65e4-46fc-a612-788ceded42e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4888f55c998f4cb7b027f12e54b75cfa'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:49.350803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-55ba6c93-65e4-46fc-a612-788ceded42e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4888f55c998f4cb7b027f12e54b75cfa'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:49.358917 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-55ba6c93-65e4-46fc-a612-788ceded42e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4888f55c998f4cb7b027f12e54b75cfa'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:49.372863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-55ba6c93-65e4-46fc-a612-788ceded42e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4888f55c998f4cb7b027f12e54b75cfa'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:49.373650 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-55ba6c93-65e4-46fc-a612-788ceded42e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4888f55c998f4cb7b027f12e54b75cfa'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:49.387172 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1298/2630] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:49 2022] DELETE /identity/v3/users/4888f55c998f4cb7b027f12e54b75cfa => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:49.690612 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b9c4c-eebc-45c2-b2d5-9156b8193d85 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:49.692122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-432b9c4c-eebc-45c2-b2d5-9156b8193d85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:49.707225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b9c4c-eebc-45c2-b2d5-9156b8193d85 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:49.912511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd01fb-5a99-43f8-88f7-827985145493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:49.913258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd01fb-5a99-43f8-88f7-827985145493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:49.913444 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd01fb-5a99-43f8-88f7-827985145493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:49.913715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd01fb-5a99-43f8-88f7-827985145493 admin admin] PATH_INFO: `/v3/projects/1a7f95a3a93e4ad6a1c1c2dc54feddad` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:49.914970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd01fb-5a99-43f8-88f7-827985145493 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a7f95a3a93e4ad6a1c1c2dc54feddad)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:49.916913 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd01fb-5a99-43f8-88f7-827985145493 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:49.960234 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:49.979682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-19dd01fb-5a99-43f8-88f7-827985145493 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a7f95a3a93e4ad6a1c1c2dc54feddad', 'request_id': 'req-19dd01fb-5a99-43f8-88f7-827985145493'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:49.986979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-19dd01fb-5a99-43f8-88f7-827985145493 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a7f95a3a93e4ad6a1c1c2dc54feddad', 'request_id': 'req-19dd01fb-5a99-43f8-88f7-827985145493'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:49.993825 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-19dd01fb-5a99-43f8-88f7-827985145493 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a7f95a3a93e4ad6a1c1c2dc54feddad', 'request_id': 'req-19dd01fb-5a99-43f8-88f7-827985145493'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:50.013237 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-19dd01fb-5a99-43f8-88f7-827985145493 admin admin] The token cache is being invalidate because project 1a7f95a3a93e4ad6a1c1c2dc54feddad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:50.013575 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-19dd01fb-5a99-43f8-88f7-827985145493 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:50.014354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-19dd01fb-5a99-43f8-88f7-827985145493 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:50.016066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1333/2631] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:49 2022] DELETE /identity/v3/projects/1a7f95a3a93e4ad6a1c1c2dc54feddad => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:50.021422 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62243e1b-834a-471d-b9c2-a48e26da291c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:50.021971 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62243e1b-834a-471d-b9c2-a48e26da291c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:50.022409 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62243e1b-834a-471d-b9c2-a48e26da291c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:50.044512 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-62243e1b-834a-471d-b9c2-a48e26da291c None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:50.045751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-62243e1b-834a-471d-b9c2-a48e26da291c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:50.120313 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1299/2632] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:50.128920 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678ea3a-c300-4b1b-b1a4-43967429b825 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:50.129136 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678ea3a-c300-4b1b-b1a4-43967429b825 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:50.129440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678ea3a-c300-4b1b-b1a4-43967429b825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:50.154194 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-7678ea3a-c300-4b1b-b1a4-43967429b825 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:50.155434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7678ea3a-c300-4b1b-b1a4-43967429b825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:50.165534 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1334/2633] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:50.170104 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62243e1b-834a-471d-b9c2-a48e26da291c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:50.180982 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62243e1b-834a-471d-b9c2-a48e26da291c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:50.186119 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b649e5a-099b-42bd-9f08-a32b080c41da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:50.186771 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b649e5a-099b-42bd-9f08-a32b080c41da admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:50.187056 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b649e5a-099b-42bd-9f08-a32b080c41da admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:50.187334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b649e5a-099b-42bd-9f08-a32b080c41da admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:50.188044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b649e5a-099b-42bd-9f08-a32b080c41da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:09:50.188350 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b649e5a-099b-42bd-9f08-a32b080c41da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:50.191324 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b649e5a-099b-42bd-9f08-a32b080c41da admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:50.203692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1300/2634] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:09:50.208952 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678ea3a-c300-4b1b-b1a4-43967429b825 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:50.218565 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678ea3a-c300-4b1b-b1a4-43967429b825 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:50.224582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abae33-5b15-41c9-83a4-addbcb5e255a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:50.225248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abae33-5b15-41c9-83a4-addbcb5e255a admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:50.225536 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abae33-5b15-41c9-83a4-addbcb5e255a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:50.225806 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abae33-5b15-41c9-83a4-addbcb5e255a admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:50.226530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97abae33-5b15-41c9-83a4-addbcb5e255a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:50.228430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97abae33-5b15-41c9-83a4-addbcb5e255a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:50.257618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1335/2635] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:50 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:09:50.262191 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b649e5a-099b-42bd-9f08-a32b080c41da admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:50.272952 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b649e5a-099b-42bd-9f08-a32b080c41da admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:50.315944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:50.334659 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92295504-22ae-4091-a49b-7fbeefcf426b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:50.335292 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92295504-22ae-4091-a49b-7fbeefcf426b admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:50.335567 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92295504-22ae-4091-a49b-7fbeefcf426b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:50.335839 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92295504-22ae-4091-a49b-7fbeefcf426b admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:50.336520 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92295504-22ae-4091-a49b-7fbeefcf426b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:50.338597 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92295504-22ae-4091-a49b-7fbeefcf426b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:50.363491 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1301/2636] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:50 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:50.367706 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abae33-5b15-41c9-83a4-addbcb5e255a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:50.378291 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abae33-5b15-41c9-83a4-addbcb5e255a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:50.383247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0524fbe-fdeb-4a08-99b6-4ffbf11fe6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:50.383904 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0524fbe-fdeb-4a08-99b6-4ffbf11fe6d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:50.384134 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0524fbe-fdeb-4a08-99b6-4ffbf11fe6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:50.384332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0524fbe-fdeb-4a08-99b6-4ffbf11fe6d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:50.385027 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0524fbe-fdeb-4a08-99b6-4ffbf11fe6d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:50.386658 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0524fbe-fdeb-4a08-99b6-4ffbf11fe6d1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:50.397978 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1336/2637] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:50.402123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92295504-22ae-4091-a49b-7fbeefcf426b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:50.413033 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92295504-22ae-4091-a49b-7fbeefcf426b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:50.418164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c0872-ae4f-40f5-b6bb-1424d17732eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:50.418798 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c0872-ae4f-40f5-b6bb-1424d17732eb admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:50.419090 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c0872-ae4f-40f5-b6bb-1424d17732eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:50.419370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c0872-ae4f-40f5-b6bb-1424d17732eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:50.420073 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c0872-ae4f-40f5-b6bb-1424d17732eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:50.421682 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c0872-ae4f-40f5-b6bb-1424d17732eb admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:50.434269 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1302/2638] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:50.439497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0524fbe-fdeb-4a08-99b6-4ffbf11fe6d1 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:50.451782 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0524fbe-fdeb-4a08-99b6-4ffbf11fe6d1 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:50.459119 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf41b5-a8a2-4e54-bf54-efe801681ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:50.459796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf41b5-a8a2-4e54-bf54-efe801681ff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:50.460102 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf41b5-a8a2-4e54-bf54-efe801681ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:50.460378 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf41b5-a8a2-4e54-bf54-efe801681ff6 admin admin] PATH_INFO: `/v3/projects/1973d505c6f44854be8f554b47897a6d/users/6ffdcfac9e1b41a8b454adb4f7897608/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:50.476840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf41b5-a8a2-4e54-bf54-efe801681ff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1973d505c6f44854be8f554b47897a6d, user_id=6ffdcfac9e1b41a8b454adb4f7897608, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:50.479520 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf41b5-a8a2-4e54-bf54-efe801681ff6 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:50.494210 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1337/2639] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:50 2022] PUT /identity/v3/projects/1973d505c6f44854be8f554b47897a6d/users/6ffdcfac9e1b41a8b454adb4f7897608/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:50.507683 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a94d49-b98f-4ee4-a58f-3fd3e4d77dd5 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:50.507970 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a94d49-b98f-4ee4-a58f-3fd3e4d77dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:50.508249 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a94d49-b98f-4ee4-a58f-3fd3e4d77dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:50.538515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-84a94d49-b98f-4ee4-a58f-3fd3e4d77dd5 None None] MFA Rules not processed for user `6ffdcfac9e1b41a8b454adb4f7897608`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:50.539760 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-84a94d49-b98f-4ee4-a58f-3fd3e4d77dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:50.607694 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1303/2640] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:50.612439 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf41b5-a8a2-4e54-bf54-efe801681ff6 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:50.678325 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf41b5-a8a2-4e54-bf54-efe801681ff6 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:50.683344 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf54eea-0876-4605-9971-2803c98f2487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:50.683976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf54eea-0876-4605-9971-2803c98f2487 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:50.684256 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf54eea-0876-4605-9971-2803c98f2487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:50.684597 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf54eea-0876-4605-9971-2803c98f2487 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:50.685218 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf54eea-0876-4605-9971-2803c98f2487 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:50.687151 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf54eea-0876-4605-9971-2803c98f2487 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:50.721503 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1338/2641] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:50 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:50.722804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a94d49-b98f-4ee4-a58f-3fd3e4d77dd5 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:50.740026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a94d49-b98f-4ee4-a58f-3fd3e4d77dd5 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:50.805259 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d0a68-7cbb-41f9-8a32-c8c8f74b3eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:50.805903 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d0a68-7cbb-41f9-8a32-c8c8f74b3eb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:50.806190 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d0a68-7cbb-41f9-8a32-c8c8f74b3eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:50.806462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d0a68-7cbb-41f9-8a32-c8c8f74b3eb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:50.807127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d0a68-7cbb-41f9-8a32-c8c8f74b3eb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:50.809056 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d0a68-7cbb-41f9-8a32-c8c8f74b3eb4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:50.835823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1304/2642] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:50 2022] POST /identity/v3/users => generated 297 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:50.839975 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf54eea-0876-4605-9971-2803c98f2487 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:50.850821 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf54eea-0876-4605-9971-2803c98f2487 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:50.855689 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28601474-459e-46f0-9a30-c7927d110c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:50.856311 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28601474-459e-46f0-9a30-c7927d110c63 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:50.856590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28601474-459e-46f0-9a30-c7927d110c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:50.856859 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28601474-459e-46f0-9a30-c7927d110c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:50.857552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28601474-459e-46f0-9a30-c7927d110c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:50.859187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28601474-459e-46f0-9a30-c7927d110c63 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:50.871053 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1339/2643] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:50.875624 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d0a68-7cbb-41f9-8a32-c8c8f74b3eb4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:50.886544 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d0a68-7cbb-41f9-8a32-c8c8f74b3eb4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:50.891456 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442b74e-588d-4655-8d7d-9d54cfb39146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:50.892145 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442b74e-588d-4655-8d7d-9d54cfb39146 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:50.892426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442b74e-588d-4655-8d7d-9d54cfb39146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:50.892714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442b74e-588d-4655-8d7d-9d54cfb39146 admin admin] PATH_INFO: `/v3/projects/841086575f5f4a7c9ab412c093db25b4/users/cd4a7bab198642d8b6ece9a4eca1c85d/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:50.909806 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442b74e-588d-4655-8d7d-9d54cfb39146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=841086575f5f4a7c9ab412c093db25b4, user_id=cd4a7bab198642d8b6ece9a4eca1c85d, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:50.912624 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442b74e-588d-4655-8d7d-9d54cfb39146 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:50.929119 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1305/2644] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:50 2022] PUT /identity/v3/projects/841086575f5f4a7c9ab412c093db25b4/users/cd4a7bab198642d8b6ece9a4eca1c85d/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:50.933300 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28601474-459e-46f0-9a30-c7927d110c63 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:50.945211 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28601474-459e-46f0-9a30-c7927d110c63 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:50.974117 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:51.016546 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647b177-883a-4b95-95bd-e10435f3883e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:51.017200 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647b177-883a-4b95-95bd-e10435f3883e admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:51.017484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647b177-883a-4b95-95bd-e10435f3883e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:51.017759 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647b177-883a-4b95-95bd-e10435f3883e admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:51.018713 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647b177-883a-4b95-95bd-e10435f3883e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:51.021258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647b177-883a-4b95-95bd-e10435f3883e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:51.032658 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1340/2645] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:51.036868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442b74e-588d-4655-8d7d-9d54cfb39146 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:51.044075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647b177-883a-4b95-95bd-e10435f3883e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:51.046847 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8647b177-883a-4b95-95bd-e10435f3883e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:51.056910 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442b74e-588d-4655-8d7d-9d54cfb39146 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:51.061520 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97adc9a1-089d-4600-916e-7a248f290a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:51.062237 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97adc9a1-089d-4600-916e-7a248f290a53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:51.062509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647b177-883a-4b95-95bd-e10435f3883e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:51.062663 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97adc9a1-089d-4600-916e-7a248f290a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:51.062778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97adc9a1-089d-4600-916e-7a248f290a53 admin admin] PATH_INFO: `/v3/projects/841086575f5f4a7c9ab412c093db25b4/users/cd4a7bab198642d8b6ece9a4eca1c85d/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:51.065509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97adc9a1-089d-4600-916e-7a248f290a53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=841086575f5f4a7c9ab412c093db25b4, user_id=cd4a7bab198642d8b6ece9a4eca1c85d, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:51.068151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97adc9a1-089d-4600-916e-7a248f290a53 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:51.068300 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855b0b8-98cb-4e54-a384-df47f84de5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:51.069009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855b0b8-98cb-4e54-a384-df47f84de5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:51.069288 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855b0b8-98cb-4e54-a384-df47f84de5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:51.069568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855b0b8-98cb-4e54-a384-df47f84de5d8 admin admin] PATH_INFO: `/v3/users/bd2b4005960b4bfb9ab7c37507e15ee0` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:51.070843 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855b0b8-98cb-4e54-a384-df47f84de5d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd2b4005960b4bfb9ab7c37507e15ee0)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:51.072948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855b0b8-98cb-4e54-a384-df47f84de5d8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:51.079621 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1306/2646] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:51 2022] PUT /identity/v3/projects/841086575f5f4a7c9ab412c093db25b4/users/cd4a7bab198642d8b6ece9a4eca1c85d/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:51.091317 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6a39a-9d66-4bdb-8c65-498c39861b32 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:51.091612 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6a39a-9d66-4bdb-8c65-498c39861b32 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:51.091918 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6a39a-9d66-4bdb-8c65-498c39861b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:51.104547 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:51.138125 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-8ad6a39a-9d66-4bdb-8c65-498c39861b32 None None] MFA Rules not processed for user `cd4a7bab198642d8b6ece9a4eca1c85d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:51.139950 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8ad6a39a-9d66-4bdb-8c65-498c39861b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:51.151399 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5855b0b8-98cb-4e54-a384-df47f84de5d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd2b4005960b4bfb9ab7c37507e15ee0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:51.152095 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5855b0b8-98cb-4e54-a384-df47f84de5d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd2b4005960b4bfb9ab7c37507e15ee0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:51.161121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5855b0b8-98cb-4e54-a384-df47f84de5d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd2b4005960b4bfb9ab7c37507e15ee0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:51.177057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5855b0b8-98cb-4e54-a384-df47f84de5d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd2b4005960b4bfb9ab7c37507e15ee0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:51.186403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5855b0b8-98cb-4e54-a384-df47f84de5d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd2b4005960b4bfb9ab7c37507e15ee0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:51.192581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1341/2647] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:51 2022] DELETE /identity/v3/users/bd2b4005960b4bfb9ab7c37507e15ee0 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:51.196925 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddb4fe-da85-4f71-93a6-146febbe837e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:51.197213 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddb4fe-da85-4f71-93a6-146febbe837e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:51.197483 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddb4fe-da85-4f71-93a6-146febbe837e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:51.217663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d3ddb4fe-da85-4f71-93a6-146febbe837e None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:51.218804 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d3ddb4fe-da85-4f71-93a6-146febbe837e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:51.222565 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1307/2648] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:51 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:09:51.233676 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad6a39a-9d66-4bdb-8c65-498c39861b32 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:51.235209 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8ad6a39a-9d66-4bdb-8c65-498c39861b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:51.289850 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1342/2649] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:09:51.301190 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddb4fe-da85-4f71-93a6-146febbe837e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:51.302874 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d3ddb4fe-da85-4f71-93a6-146febbe837e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:51.304634 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad6a39a-9d66-4bdb-8c65-498c39861b32 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:51.347521 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:51.356786 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a54e39-16ee-43df-ada5-9c00d3f508ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:51.357593 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a54e39-16ee-43df-ada5-9c00d3f508ab service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:51.357996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a54e39-16ee-43df-ada5-9c00d3f508ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:51.358387 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a54e39-16ee-43df-ada5-9c00d3f508ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:51.368343 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddb4fe-da85-4f71-93a6-146febbe837e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:51.373018 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f7aa8-ebbb-44a8-b91b-21490b22152e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:51.373643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f7aa8-ebbb-44a8-b91b-21490b22152e service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:51.373952 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f7aa8-ebbb-44a8-b91b-21490b22152e service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:51.374317 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f7aa8-ebbb-44a8-b91b-21490b22152e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:51.384277 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f7aa8-ebbb-44a8-b91b-21490b22152e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:51.385856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f7aa8-ebbb-44a8-b91b-21490b22152e service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:51.396610 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1343/2650] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:09:51.426563 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:51.432376 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a54e39-16ee-43df-ada5-9c00d3f508ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:51.433876 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a54e39-16ee-43df-ada5-9c00d3f508ab service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:51.445662 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1308/2651] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:09:51.563873 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f7aa8-ebbb-44a8-b91b-21490b22152e service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:51.573729 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f7aa8-ebbb-44a8-b91b-21490b22152e service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:51.577410 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1cbe3-0683-4dd7-864a-3589ffa2a63d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:51.578089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1cbe3-0683-4dd7-864a-3589ffa2a63d service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:51.578466 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1cbe3-0683-4dd7-864a-3589ffa2a63d service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:51.578737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1cbe3-0683-4dd7-864a-3589ffa2a63d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:51.580594 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0aa1cbe3-0683-4dd7-864a-3589ffa2a63d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:51.653275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1cbe3-0683-4dd7-864a-3589ffa2a63d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:51.654805 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1cbe3-0683-4dd7-864a-3589ffa2a63d service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:51.665421 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1344/2652] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:51.712124 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a54e39-16ee-43df-ada5-9c00d3f508ab service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:51.713980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-01a54e39-16ee-43df-ada5-9c00d3f508ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:51.750838 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a54e39-16ee-43df-ada5-9c00d3f508ab service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:51.763119 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7beac-5e34-4851-852b-e62f5704ff4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:51.763784 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7beac-5e34-4851-852b-e62f5704ff4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:51.764068 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7beac-5e34-4851-852b-e62f5704ff4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:51.764339 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7beac-5e34-4851-852b-e62f5704ff4b admin admin] PATH_INFO: `/v3/projects/4ac863e436fc44199b9e6d7d952b973f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:51.765512 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7beac-5e34-4851-852b-e62f5704ff4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ac863e436fc44199b9e6d7d952b973f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:51.768011 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7beac-5e34-4851-852b-e62f5704ff4b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:51.812238 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1cbe3-0683-4dd7-864a-3589ffa2a63d service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:51.814107 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0aa1cbe3-0683-4dd7-864a-3589ffa2a63d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:51.829850 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1cbe3-0683-4dd7-864a-3589ffa2a63d service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:51.837515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9ba7beac-5e34-4851-852b-e62f5704ff4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ac863e436fc44199b9e6d7d952b973f', 'request_id': 'req-9ba7beac-5e34-4851-852b-e62f5704ff4b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:51.848759 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9ba7beac-5e34-4851-852b-e62f5704ff4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ac863e436fc44199b9e6d7d952b973f', 'request_id': 'req-9ba7beac-5e34-4851-852b-e62f5704ff4b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:51.857028 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9ba7beac-5e34-4851-852b-e62f5704ff4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ac863e436fc44199b9e6d7d952b973f', 'request_id': 'req-9ba7beac-5e34-4851-852b-e62f5704ff4b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:51.876822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9ba7beac-5e34-4851-852b-e62f5704ff4b admin admin] The token cache is being invalidate because project 4ac863e436fc44199b9e6d7d952b973f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:51.877423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9ba7beac-5e34-4851-852b-e62f5704ff4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:51.878305 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9ba7beac-5e34-4851-852b-e62f5704ff4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:51.880336 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1309/2653] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:51 2022] DELETE /identity/v3/projects/4ac863e436fc44199b9e6d7d952b973f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:51.885775 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29981c2-add2-43ef-87f8-0031bc7b4f4d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:51.886124 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29981c2-add2-43ef-87f8-0031bc7b4f4d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:51.886744 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29981c2-add2-43ef-87f8-0031bc7b4f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:51.908110 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dca426-c1af-4f34-b777-4c557be0192d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:51.909012 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dca426-c1af-4f34-b777-4c557be0192d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:51.909468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dca426-c1af-4f34-b777-4c557be0192d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:51.909970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dca426-c1af-4f34-b777-4c557be0192d admin admin] PATH_INFO: `/v3/users/c7d7122be44442e4bdae4511e9ff7ec0` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:51.911742 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dca426-c1af-4f34-b777-4c557be0192d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7d7122be44442e4bdae4511e9ff7ec0)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:51.914658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c29981c2-add2-43ef-87f8-0031bc7b4f4d None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:51.915187 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dca426-c1af-4f34-b777-4c557be0192d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:51.916650 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c29981c2-add2-43ef-87f8-0031bc7b4f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:51.932222 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1310/2654] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:09:51.941480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd8667-3e88-4acb-a22c-f977dd5d2f54 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:51.941860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd8667-3e88-4acb-a22c-f977dd5d2f54 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:51.942221 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd8667-3e88-4acb-a22c-f977dd5d2f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:51.986283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-58cd8667-3e88-4acb-a22c-f977dd5d2f54 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:51.992233 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-58cd8667-3e88-4acb-a22c-f977dd5d2f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:51.995852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1311/2655] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:51.999051 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-28dca426-c1af-4f34-b777-4c557be0192d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7d7122be44442e4bdae4511e9ff7ec0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:52.000102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd8667-3e88-4acb-a22c-f977dd5d2f54 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:52.001112 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-28dca426-c1af-4f34-b777-4c557be0192d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7d7122be44442e4bdae4511e9ff7ec0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:52.011374 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd8667-3e88-4acb-a22c-f977dd5d2f54 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:52.015059 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-28dca426-c1af-4f34-b777-4c557be0192d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7d7122be44442e4bdae4511e9ff7ec0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:52.017984 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b5e8a-d0da-4992-9cb4-f77b4325e8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:52.018654 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b5e8a-d0da-4992-9cb4-f77b4325e8f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:52.018948 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b5e8a-d0da-4992-9cb4-f77b4325e8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:52.019333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b5e8a-d0da-4992-9cb4-f77b4325e8f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:52.020714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b5e8a-d0da-4992-9cb4-f77b4325e8f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:09:52.021435 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b5e8a-d0da-4992-9cb4-f77b4325e8f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:52.024225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b5e8a-d0da-4992-9cb4-f77b4325e8f3 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:52.039657 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-28dca426-c1af-4f34-b777-4c557be0192d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7d7122be44442e4bdae4511e9ff7ec0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:52.048668 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-28dca426-c1af-4f34-b777-4c557be0192d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7d7122be44442e4bdae4511e9ff7ec0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:52.060825 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1345/2656] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:51 2022] DELETE /identity/v3/users/c7d7122be44442e4bdae4511e9ff7ec0 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:52.062312 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:52.063022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1312/2657] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:09:52.074687 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dca426-c1af-4f34-b777-4c557be0192d admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:52.075409 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6e793-046c-4510-9607-ad2a27edf2eb None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:52.075710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6e793-046c-4510-9607-ad2a27edf2eb None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:52.076033 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6e793-046c-4510-9607-ad2a27edf2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:52.078689 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-28dca426-c1af-4f34-b777-4c557be0192d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:52.102902 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a8a6e793-046c-4510-9607-ad2a27edf2eb None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:52.104591 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a8a6e793-046c-4510-9607-ad2a27edf2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:52.157155 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dca426-c1af-4f34-b777-4c557be0192d admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:52.162160 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebb8ff-bf37-4292-8993-5fde2efd5b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:52.162784 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebb8ff-bf37-4292-8993-5fde2efd5b2b admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:52.163062 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebb8ff-bf37-4292-8993-5fde2efd5b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:52.163331 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebb8ff-bf37-4292-8993-5fde2efd5b2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:52.163996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebb8ff-bf37-4292-8993-5fde2efd5b2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:52.167299 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebb8ff-bf37-4292-8993-5fde2efd5b2b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:52.202678 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1313/2658] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:52.210781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1346/2659] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:52 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:52.213338 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebb8ff-bf37-4292-8993-5fde2efd5b2b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:52.213455 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6e793-046c-4510-9607-ad2a27edf2eb None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:52.216388 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b4ebb8ff-bf37-4292-8993-5fde2efd5b2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:52.226290 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6e793-046c-4510-9607-ad2a27edf2eb None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:52.287897 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebb8ff-bf37-4292-8993-5fde2efd5b2b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:52.292049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34933e4-d4a8-4c56-81eb-ef15d070c174 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:52.292705 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34933e4-d4a8-4c56-81eb-ef15d070c174 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:52.292991 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34933e4-d4a8-4c56-81eb-ef15d070c174 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:52.293267 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34933e4-d4a8-4c56-81eb-ef15d070c174 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:52.380186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34933e4-d4a8-4c56-81eb-ef15d070c174 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:52.381843 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34933e4-d4a8-4c56-81eb-ef15d070c174 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:52.394921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1347/2660] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:52.417808 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:52.435346 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e8e04-6a7a-4185-9650-f92b8e35a2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:52.435999 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e8e04-6a7a-4185-9650-f92b8e35a2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:52.436575 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e8e04-6a7a-4185-9650-f92b8e35a2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:52.436992 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e8e04-6a7a-4185-9650-f92b8e35a2b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:52.437863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e8e04-6a7a-4185-9650-f92b8e35a2b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:52.441043 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e8e04-6a7a-4185-9650-f92b8e35a2b6 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:52.470957 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1314/2661] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:52 2022] POST /identity/v3/users => generated 288 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:52.475017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34933e4-d4a8-4c56-81eb-ef15d070c174 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:52.486196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34933e4-d4a8-4c56-81eb-ef15d070c174 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:52.492129 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033949a7-e808-4a98-ae9d-3ec7ac7278ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:52.492777 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033949a7-e808-4a98-ae9d-3ec7ac7278ac admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:52.493057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033949a7-e808-4a98-ae9d-3ec7ac7278ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:52.493337 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033949a7-e808-4a98-ae9d-3ec7ac7278ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:52.494072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033949a7-e808-4a98-ae9d-3ec7ac7278ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:52.495671 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033949a7-e808-4a98-ae9d-3ec7ac7278ac admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:52.509966 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1348/2662] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:52.514467 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e8e04-6a7a-4185-9650-f92b8e35a2b6 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:52.525377 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e8e04-6a7a-4185-9650-f92b8e35a2b6 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:52.530926 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989816e5-acce-4a2c-89fc-2e769628a038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:52.531568 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989816e5-acce-4a2c-89fc-2e769628a038 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:52.531851 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989816e5-acce-4a2c-89fc-2e769628a038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:52.532129 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989816e5-acce-4a2c-89fc-2e769628a038 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:52.532912 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989816e5-acce-4a2c-89fc-2e769628a038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:52.534658 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989816e5-acce-4a2c-89fc-2e769628a038 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:52.545680 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1315/2663] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:09:52.550490 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033949a7-e808-4a98-ae9d-3ec7ac7278ac admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:52.564514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033949a7-e808-4a98-ae9d-3ec7ac7278ac admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:52.571731 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d9b54-af0b-472f-82db-aefa2e062c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:52.572233 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d9b54-af0b-472f-82db-aefa2e062c0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:52.572584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d9b54-af0b-472f-82db-aefa2e062c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:52.572928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d9b54-af0b-472f-82db-aefa2e062c0e admin admin] PATH_INFO: `/v3/projects/09b2794eca824cffae1b872da0beacea/users/1b4b80ab48bc4dab83fd03677544925c/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:52.590897 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d9b54-af0b-472f-82db-aefa2e062c0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=09b2794eca824cffae1b872da0beacea, user_id=1b4b80ab48bc4dab83fd03677544925c, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:52.593705 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d9b54-af0b-472f-82db-aefa2e062c0e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:52.602285 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:52.609643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1349/2664] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:52 2022] PUT /identity/v3/projects/09b2794eca824cffae1b872da0beacea/users/1b4b80ab48bc4dab83fd03677544925c/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:52.621703 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3cc12-f491-420e-b374-b61c5634244b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:52.622031 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3cc12-f491-420e-b374-b61c5634244b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:52.622297 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3cc12-f491-420e-b374-b61c5634244b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:52.656663 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d7a3cc12-f491-420e-b374-b61c5634244b None None] MFA Rules not processed for user `1b4b80ab48bc4dab83fd03677544925c`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:52.658762 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d7a3cc12-f491-420e-b374-b61c5634244b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:52.704140 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d9b54-af0b-472f-82db-aefa2e062c0e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:52.710044 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-282d9b54-af0b-472f-82db-aefa2e062c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:52.757935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1316/2665] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:52.763316 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3cc12-f491-420e-b374-b61c5634244b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:52.795866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d9b54-af0b-472f-82db-aefa2e062c0e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:52.801584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2f40a-a892-4b2a-9117-9da6297bbb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:52.802292 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2f40a-a892-4b2a-9117-9da6297bbb86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:52.802578 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2f40a-a892-4b2a-9117-9da6297bbb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:52.802846 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2f40a-a892-4b2a-9117-9da6297bbb86 admin admin] PATH_INFO: `/v3/projects/9410b1a201fd41b485489a8bff5cbfa3` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:52.804198 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e2f40a-a892-4b2a-9117-9da6297bbb86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9410b1a201fd41b485489a8bff5cbfa3)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:52.806138 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e2f40a-a892-4b2a-9117-9da6297bbb86 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:52.834807 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3cc12-f491-420e-b374-b61c5634244b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:52.839867 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff448f2-72e2-4275-b5ae-9bcf4f2bfd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:52.840508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff448f2-72e2-4275-b5ae-9bcf4f2bfd82 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:52.840792 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff448f2-72e2-4275-b5ae-9bcf4f2bfd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:52.841072 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff448f2-72e2-4275-b5ae-9bcf4f2bfd82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:52.841981 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff448f2-72e2-4275-b5ae-9bcf4f2bfd82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:52.844085 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff448f2-72e2-4275-b5ae-9bcf4f2bfd82 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:52.863667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e5e2f40a-a892-4b2a-9117-9da6297bbb86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9410b1a201fd41b485489a8bff5cbfa3', 'request_id': 'req-e5e2f40a-a892-4b2a-9117-9da6297bbb86'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:52.870691 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e5e2f40a-a892-4b2a-9117-9da6297bbb86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9410b1a201fd41b485489a8bff5cbfa3', 'request_id': 'req-e5e2f40a-a892-4b2a-9117-9da6297bbb86'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:52.876168 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1317/2666] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:52 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:52.877805 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e5e2f40a-a892-4b2a-9117-9da6297bbb86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9410b1a201fd41b485489a8bff5cbfa3', 'request_id': 'req-e5e2f40a-a892-4b2a-9117-9da6297bbb86'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:52.880843 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff448f2-72e2-4275-b5ae-9bcf4f2bfd82 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:52.891792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e5e2f40a-a892-4b2a-9117-9da6297bbb86 admin admin] The token cache is being invalidate because project 9410b1a201fd41b485489a8bff5cbfa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:52.892079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e5e2f40a-a892-4b2a-9117-9da6297bbb86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:52.892763 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e5e2f40a-a892-4b2a-9117-9da6297bbb86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:52.894538 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1350/2667] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:52 2022] DELETE /identity/v3/projects/9410b1a201fd41b485489a8bff5cbfa3 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:52.898522 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2f40a-a892-4b2a-9117-9da6297bbb86 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:52.900039 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e5e2f40a-a892-4b2a-9117-9da6297bbb86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:52.940416 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:52.960726 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff448f2-72e2-4275-b5ae-9bcf4f2bfd82 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:52.965665 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78511c8-d4bf-4bdd-85df-d041782ebafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:52.966351 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78511c8-d4bf-4bdd-85df-d041782ebafe admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:52.966634 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78511c8-d4bf-4bdd-85df-d041782ebafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:52.966910 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78511c8-d4bf-4bdd-85df-d041782ebafe admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:52.967578 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78511c8-d4bf-4bdd-85df-d041782ebafe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:52.969507 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78511c8-d4bf-4bdd-85df-d041782ebafe admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:52.989492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2f40a-a892-4b2a-9117-9da6297bbb86 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:52.994733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1318/2668] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:52 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:52.997953 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a41375-1811-455a-b9f6-de06d0907d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:52.998642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a41375-1811-455a-b9f6-de06d0907d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:52.998932 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78511c8-d4bf-4bdd-85df-d041782ebafe admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:52.999148 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a41375-1811-455a-b9f6-de06d0907d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:52.999148 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a41375-1811-455a-b9f6-de06d0907d0e admin admin] PATH_INFO: `/v3/users/0741bd92dabb4cbb84ea8a22d8013285` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:53.000499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a41375-1811-455a-b9f6-de06d0907d0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0741bd92dabb4cbb84ea8a22d8013285)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:53.000769 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b78511c8-d4bf-4bdd-85df-d041782ebafe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:53.002465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a41375-1811-455a-b9f6-de06d0907d0e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:53.018270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78511c8-d4bf-4bdd-85df-d041782ebafe admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:53.023199 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b3c73-d660-424b-960b-8051194217c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:53.023850 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b3c73-d660-424b-960b-8051194217c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:53.024139 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b3c73-d660-424b-960b-8051194217c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:53.024417 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b3c73-d660-424b-960b-8051194217c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:53.025219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b3c73-d660-424b-960b-8051194217c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:53.027193 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b3c73-d660-424b-960b-8051194217c1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:53.040906 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:53.041362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1319/2669] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:09:53.046144 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b3c73-d660-424b-960b-8051194217c1 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:53.060070 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b3c73-d660-424b-960b-8051194217c1 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:53.065186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-38a41375-1811-455a-b9f6-de06d0907d0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0741bd92dabb4cbb84ea8a22d8013285'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:53.065794 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1546b3-81f2-4a80-8856-d17e5b1ffa8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:53.066417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-38a41375-1811-455a-b9f6-de06d0907d0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0741bd92dabb4cbb84ea8a22d8013285'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:53.066824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1546b3-81f2-4a80-8856-d17e5b1ffa8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:53.067633 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1546b3-81f2-4a80-8856-d17e5b1ffa8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:53.067823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1546b3-81f2-4a80-8856-d17e5b1ffa8c admin admin] PATH_INFO: `/v3/projects/b648ab031a90453c935145efba14ee56/users/fb8c73db9afc4d7d900e26e016ae1bba/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:53.076685 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-38a41375-1811-455a-b9f6-de06d0907d0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0741bd92dabb4cbb84ea8a22d8013285'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:53.090621 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1546b3-81f2-4a80-8856-d17e5b1ffa8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b648ab031a90453c935145efba14ee56, user_id=fb8c73db9afc4d7d900e26e016ae1bba, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:53.093317 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1546b3-81f2-4a80-8856-d17e5b1ffa8c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:53.096307 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-38a41375-1811-455a-b9f6-de06d0907d0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0741bd92dabb4cbb84ea8a22d8013285'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:53.105272 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-38a41375-1811-455a-b9f6-de06d0907d0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0741bd92dabb4cbb84ea8a22d8013285'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:53.108885 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1320/2670] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:53 2022] PUT /identity/v3/projects/b648ab031a90453c935145efba14ee56/users/fb8c73db9afc4d7d900e26e016ae1bba/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:53.114684 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1546b3-81f2-4a80-8856-d17e5b1ffa8c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:53.115471 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1351/2671] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:52 2022] DELETE /identity/v3/users/0741bd92dabb4cbb84ea8a22d8013285 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:53.117225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3a1546b3-81f2-4a80-8856-d17e5b1ffa8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:53.224826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1546b3-81f2-4a80-8856-d17e5b1ffa8c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:53.342185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94919f3-6853-41d4-970b-4198557b4ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:53.342844 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94919f3-6853-41d4-970b-4198557b4ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:53.343145 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94919f3-6853-41d4-970b-4198557b4ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:53.343407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94919f3-6853-41d4-970b-4198557b4ae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:53.344762 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94919f3-6853-41d4-970b-4198557b4ae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:53.347311 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94919f3-6853-41d4-970b-4198557b4ae8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:53.359401 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1321/2672] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:09:53.363840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a41375-1811-455a-b9f6-de06d0907d0e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:53.377355 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a41375-1811-455a-b9f6-de06d0907d0e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:53.382688 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe38d1-1fec-4b82-8764-831dba5f27ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:53.384517 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe38d1-1fec-4b82-8764-831dba5f27ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:53.384517 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe38d1-1fec-4b82-8764-831dba5f27ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:53.384989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe38d1-1fec-4b82-8764-831dba5f27ab admin admin] PATH_INFO: `/v3/projects/b648ab031a90453c935145efba14ee56/users/fb8c73db9afc4d7d900e26e016ae1bba/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:53.387648 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe38d1-1fec-4b82-8764-831dba5f27ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=b648ab031a90453c935145efba14ee56, user_id=fb8c73db9afc4d7d900e26e016ae1bba, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:53.390408 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe38d1-1fec-4b82-8764-831dba5f27ab admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:53.404263 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1352/2673] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:53 2022] PUT /identity/v3/projects/b648ab031a90453c935145efba14ee56/users/fb8c73db9afc4d7d900e26e016ae1bba/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:53.417030 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaec9f2-34f2-47f4-8c7f-52d71324fbc2 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:53.417030 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaec9f2-34f2-47f4-8c7f-52d71324fbc2 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:53.417030 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaec9f2-34f2-47f4-8c7f-52d71324fbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:53.473669 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-caaec9f2-34f2-47f4-8c7f-52d71324fbc2 None None] MFA Rules not processed for user `fb8c73db9afc4d7d900e26e016ae1bba`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:53.474756 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-caaec9f2-34f2-47f4-8c7f-52d71324fbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:53.513628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:53.562160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1322/2674] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:53 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:53.574024 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe38d1-1fec-4b82-8764-831dba5f27ab admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:53.575811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d3fe38d1-1fec-4b82-8764-831dba5f27ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:53.642639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe38d1-1fec-4b82-8764-831dba5f27ab admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:53.646463 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de939cff-1682-4753-a666-fae851cfb4e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:53.647097 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de939cff-1682-4753-a666-fae851cfb4e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:53.647404 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de939cff-1682-4753-a666-fae851cfb4e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:53.647710 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de939cff-1682-4753-a666-fae851cfb4e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:53.649594 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-de939cff-1682-4753-a666-fae851cfb4e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:53.728635 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaec9f2-34f2-47f4-8c7f-52d71324fbc2 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:53.729374 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de939cff-1682-4753-a666-fae851cfb4e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:53.730091 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-caaec9f2-34f2-47f4-8c7f-52d71324fbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:53.730886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de939cff-1682-4753-a666-fae851cfb4e9 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:53.743130 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1353/2675] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:53.811566 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaec9f2-34f2-47f4-8c7f-52d71324fbc2 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:53.817693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8ebd4-dd82-4f97-abc2-72f6d2a5b666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:53.818389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8ebd4-dd82-4f97-abc2-72f6d2a5b666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:53.818674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8ebd4-dd82-4f97-abc2-72f6d2a5b666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:53.818950 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8ebd4-dd82-4f97-abc2-72f6d2a5b666 admin admin] PATH_INFO: `/v3/projects/163d33c680644f08bbba06fe1b7d7365` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:53.820178 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c8ebd4-dd82-4f97-abc2-72f6d2a5b666 admin admin] RBAC: Authorizing `identity:delete_project(project_id=163d33c680644f08bbba06fe1b7d7365)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:53.822130 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c8ebd4-dd82-4f97-abc2-72f6d2a5b666 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:53.875964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-25c8ebd4-dd82-4f97-abc2-72f6d2a5b666 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '163d33c680644f08bbba06fe1b7d7365', 'request_id': 'req-25c8ebd4-dd82-4f97-abc2-72f6d2a5b666'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:53.885346 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-25c8ebd4-dd82-4f97-abc2-72f6d2a5b666 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '163d33c680644f08bbba06fe1b7d7365', 'request_id': 'req-25c8ebd4-dd82-4f97-abc2-72f6d2a5b666'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:53.891694 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-25c8ebd4-dd82-4f97-abc2-72f6d2a5b666 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '163d33c680644f08bbba06fe1b7d7365', 'request_id': 'req-25c8ebd4-dd82-4f97-abc2-72f6d2a5b666'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:53.908275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-25c8ebd4-dd82-4f97-abc2-72f6d2a5b666 admin admin] The token cache is being invalidate because project 163d33c680644f08bbba06fe1b7d7365 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:53.908616 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-25c8ebd4-dd82-4f97-abc2-72f6d2a5b666 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:53.909355 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-25c8ebd4-dd82-4f97-abc2-72f6d2a5b666 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:53.910984 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1323/2676] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:53 2022] DELETE /identity/v3/projects/163d33c680644f08bbba06fe1b7d7365 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:53.919243 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121befc-e1f0-411a-bdf2-caaaeff8a05e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:53.920370 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121befc-e1f0-411a-bdf2-caaaeff8a05e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:53.920685 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121befc-e1f0-411a-bdf2-caaaeff8a05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:53.946135 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f121befc-e1f0-411a-bdf2-caaaeff8a05e None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:53.947371 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f121befc-e1f0-411a-bdf2-caaaeff8a05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:54.007213 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:54.045550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1354/2677] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:09:54.054554 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683efbf-1591-4872-be38-f0c51d344a8f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:54.054922 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683efbf-1591-4872-be38-f0c51d344a8f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:54.055271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683efbf-1591-4872-be38-f0c51d344a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:54.086294 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6683efbf-1591-4872-be38-f0c51d344a8f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:54.087617 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6683efbf-1591-4872-be38-f0c51d344a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:54.102765 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1324/2678] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:54.107339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f121befc-e1f0-411a-bdf2-caaaeff8a05e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:54.119164 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f121befc-e1f0-411a-bdf2-caaaeff8a05e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:54.124199 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae709cf-ed84-4e3d-bcc1-21da910535e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:54.124855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae709cf-ed84-4e3d-bcc1-21da910535e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:54.125132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae709cf-ed84-4e3d-bcc1-21da910535e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:54.125402 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae709cf-ed84-4e3d-bcc1-21da910535e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:54.126217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae709cf-ed84-4e3d-bcc1-21da910535e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:09:54.127016 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae709cf-ed84-4e3d-bcc1-21da910535e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:54.128672 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae709cf-ed84-4e3d-bcc1-21da910535e8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:54.145748 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1355/2679] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:09:54.150691 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683efbf-1591-4872-be38-f0c51d344a8f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:54.160822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683efbf-1591-4872-be38-f0c51d344a8f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:54.165945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d59c9d-95c6-4895-ab13-7e4298c3ef71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:54.166734 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d59c9d-95c6-4895-ab13-7e4298c3ef71 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:54.167025 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d59c9d-95c6-4895-ab13-7e4298c3ef71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:54.167303 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d59c9d-95c6-4895-ab13-7e4298c3ef71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:54.167998 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d59c9d-95c6-4895-ab13-7e4298c3ef71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:54.170006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d59c9d-95c6-4895-ab13-7e4298c3ef71 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:54.206252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1325/2680] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:54 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:54.210587 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae709cf-ed84-4e3d-bcc1-21da910535e8 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:54.221403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae709cf-ed84-4e3d-bcc1-21da910535e8 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:54.286306 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e0cf5-2308-4a73-8842-a090a9d6a436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:54.286865 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e0cf5-2308-4a73-8842-a090a9d6a436 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:54.287149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e0cf5-2308-4a73-8842-a090a9d6a436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:54.287427 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e0cf5-2308-4a73-8842-a090a9d6a436 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:54.288165 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e0cf5-2308-4a73-8842-a090a9d6a436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:54.290100 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e0cf5-2308-4a73-8842-a090a9d6a436 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:54.318701 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1356/2681] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:54 2022] POST /identity/v3/users => generated 285 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:09:54.320150 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d59c9d-95c6-4895-ab13-7e4298c3ef71 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:54.333048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d59c9d-95c6-4895-ab13-7e4298c3ef71 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:54.338340 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b838d4-586d-406c-8468-b745fe46d076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:54.338983 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b838d4-586d-406c-8468-b745fe46d076 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:54.339267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b838d4-586d-406c-8468-b745fe46d076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:54.339544 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b838d4-586d-406c-8468-b745fe46d076 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:54.340254 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b838d4-586d-406c-8468-b745fe46d076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:54.341965 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b838d4-586d-406c-8468-b745fe46d076 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:54.353785 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1326/2682] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:54.359202 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e0cf5-2308-4a73-8842-a090a9d6a436 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:54.370888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e0cf5-2308-4a73-8842-a090a9d6a436 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:54.380697 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c961cfc-b9eb-42ad-97a8-54b63f04aefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:54.381358 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c961cfc-b9eb-42ad-97a8-54b63f04aefc admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:54.381694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c961cfc-b9eb-42ad-97a8-54b63f04aefc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:54.382000 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c961cfc-b9eb-42ad-97a8-54b63f04aefc admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:54.382702 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c961cfc-b9eb-42ad-97a8-54b63f04aefc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:54.384457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c961cfc-b9eb-42ad-97a8-54b63f04aefc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:54.397283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1357/2683] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:54.401618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b838d4-586d-406c-8468-b745fe46d076 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:54.412459 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b838d4-586d-406c-8468-b745fe46d076 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:54.417114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce50373-0b89-4be6-be14-d677da1cfa65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:54.417775 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce50373-0b89-4be6-be14-d677da1cfa65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:54.418094 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce50373-0b89-4be6-be14-d677da1cfa65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:54.418366 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce50373-0b89-4be6-be14-d677da1cfa65 admin admin] PATH_INFO: `/v3/projects/030a3645d6d24ed4b10bd4b423b0f24b/users/1510e52b41f545e4b61d310a08c23a3e/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:54.434578 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce50373-0b89-4be6-be14-d677da1cfa65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=030a3645d6d24ed4b10bd4b423b0f24b, user_id=1510e52b41f545e4b61d310a08c23a3e, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:54.437222 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce50373-0b89-4be6-be14-d677da1cfa65 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:54.439625 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:54.453407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1327/2684] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:54 2022] PUT /identity/v3/projects/030a3645d6d24ed4b10bd4b423b0f24b/users/1510e52b41f545e4b61d310a08c23a3e/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:54.467157 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bea7e-0ba1-43fc-95c5-34dca6656bb3 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:54.467519 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bea7e-0ba1-43fc-95c5-34dca6656bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:54.467736 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bea7e-0ba1-43fc-95c5-34dca6656bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:54.499960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1b7bea7e-0ba1-43fc-95c5-34dca6656bb3 None None] MFA Rules not processed for user `1510e52b41f545e4b61d310a08c23a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:54.501455 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1b7bea7e-0ba1-43fc-95c5-34dca6656bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:54.515492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce50373-0b89-4be6-be14-d677da1cfa65 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:54.517663 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2ce50373-0b89-4be6-be14-d677da1cfa65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:54.576515 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1358/2685] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:54 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:54.589009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7bea7e-0ba1-43fc-95c5-34dca6656bb3 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:54.590426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1b7bea7e-0ba1-43fc-95c5-34dca6656bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:54.602411 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce50373-0b89-4be6-be14-d677da1cfa65 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:54.607970 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e783057-b48c-4c3f-8f5e-ed048b9baef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:54.608638 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e783057-b48c-4c3f-8f5e-ed048b9baef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:54.608928 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e783057-b48c-4c3f-8f5e-ed048b9baef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:54.609239 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e783057-b48c-4c3f-8f5e-ed048b9baef7 admin admin] PATH_INFO: `/v3/users/6ffdcfac9e1b41a8b454adb4f7897608` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:54.610522 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e783057-b48c-4c3f-8f5e-ed048b9baef7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ffdcfac9e1b41a8b454adb4f7897608)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:54.613632 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e783057-b48c-4c3f-8f5e-ed048b9baef7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:54.638802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:54.654860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7bea7e-0ba1-43fc-95c5-34dca6656bb3 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:54.659893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43241b-5cd9-4994-aca2-bcf36543db8b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69d216a568b546589e854c2b4476efe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:54.660530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43241b-5cd9-4994-aca2-bcf36543db8b service glance] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:54.660810 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43241b-5cd9-4994-aca2-bcf36543db8b service glance] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:54.661081 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43241b-5cd9-4994-aca2-bcf36543db8b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:54.665637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3e783057-b48c-4c3f-8f5e-ed048b9baef7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ffdcfac9e1b41a8b454adb4f7897608'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:54.666753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3e783057-b48c-4c3f-8f5e-ed048b9baef7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ffdcfac9e1b41a8b454adb4f7897608'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:54.670970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d43241b-5cd9-4994-aca2-bcf36543db8b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:54.672727 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d43241b-5cd9-4994-aca2-bcf36543db8b service glance] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:54.674978 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3e783057-b48c-4c3f-8f5e-ed048b9baef7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ffdcfac9e1b41a8b454adb4f7897608'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:54.684419 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1359/2686] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:54 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:09:54.691437 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3e783057-b48c-4c3f-8f5e-ed048b9baef7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ffdcfac9e1b41a8b454adb4f7897608'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:54.692524 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3e783057-b48c-4c3f-8f5e-ed048b9baef7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ffdcfac9e1b41a8b454adb4f7897608'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:54.711985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1328/2687] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:54 2022] DELETE /identity/v3/users/6ffdcfac9e1b41a8b454adb4f7897608 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:54.716495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e308331-6613-4ad3-97a8-740d1cda4138 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:54.716785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e308331-6613-4ad3-97a8-740d1cda4138 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:54.717065 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e308331-6613-4ad3-97a8-740d1cda4138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:54.738693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-3e308331-6613-4ad3-97a8-740d1cda4138 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:54.740012 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3e308331-6613-4ad3-97a8-740d1cda4138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:54.814304 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1360/2688] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:54.822947 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e783057-b48c-4c3f-8f5e-ed048b9baef7 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:54.824483 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3e783057-b48c-4c3f-8f5e-ed048b9baef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:54.892508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e783057-b48c-4c3f-8f5e-ed048b9baef7 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:54.896423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb0850-5b53-4929-8217-79436ff0f901 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:54.897050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb0850-5b53-4929-8217-79436ff0f901 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:54.897341 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb0850-5b53-4929-8217-79436ff0f901 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:54.897619 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb0850-5b53-4929-8217-79436ff0f901 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:54.915968 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb0850-5b53-4929-8217-79436ff0f901 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:54.918114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb0850-5b53-4929-8217-79436ff0f901 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:54.933350 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1329/2689] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:55.303533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e308331-6613-4ad3-97a8-740d1cda4138 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:55.305025 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3e308331-6613-4ad3-97a8-740d1cda4138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:55.331510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e308331-6613-4ad3-97a8-740d1cda4138 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:55.341860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbf8e1-1fe5-4536-bbed-a3a6b4e182fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:55.343714 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbf8e1-1fe5-4536-bbed-a3a6b4e182fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:55.344221 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbf8e1-1fe5-4536-bbed-a3a6b4e182fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:55.344818 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbf8e1-1fe5-4536-bbed-a3a6b4e182fa admin admin] PATH_INFO: `/v3/projects/1973d505c6f44854be8f554b47897a6d` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:55.349361 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbf8e1-1fe5-4536-bbed-a3a6b4e182fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1973d505c6f44854be8f554b47897a6d)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:55.352470 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbf8e1-1fe5-4536-bbed-a3a6b4e182fa admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:55.432259 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0cfbf8e1-1fe5-4536-bbed-a3a6b4e182fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1973d505c6f44854be8f554b47897a6d', 'request_id': 'req-0cfbf8e1-1fe5-4536-bbed-a3a6b4e182fa'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:55.440918 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0cfbf8e1-1fe5-4536-bbed-a3a6b4e182fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1973d505c6f44854be8f554b47897a6d', 'request_id': 'req-0cfbf8e1-1fe5-4536-bbed-a3a6b4e182fa'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:55.449287 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0cfbf8e1-1fe5-4536-bbed-a3a6b4e182fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1973d505c6f44854be8f554b47897a6d', 'request_id': 'req-0cfbf8e1-1fe5-4536-bbed-a3a6b4e182fa'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:55.465664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0cfbf8e1-1fe5-4536-bbed-a3a6b4e182fa admin admin] The token cache is being invalidate because project 1973d505c6f44854be8f554b47897a6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:55.466023 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0cfbf8e1-1fe5-4536-bbed-a3a6b4e182fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:55.466986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0cfbf8e1-1fe5-4536-bbed-a3a6b4e182fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:55.468766 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1361/2690] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:55 2022] DELETE /identity/v3/projects/1973d505c6f44854be8f554b47897a6d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:55.473208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb0850-5b53-4929-8217-79436ff0f901 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:55.489142 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb0850-5b53-4929-8217-79436ff0f901 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:55.506788 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:55.562528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd8e7c-db54-4cc1-9621-680c65df1fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:55.563221 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd8e7c-db54-4cc1-9621-680c65df1fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:55.563465 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd8e7c-db54-4cc1-9621-680c65df1fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:55.564370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd8e7c-db54-4cc1-9621-680c65df1fdb admin admin] PATH_INFO: `/v3/users/cd4a7bab198642d8b6ece9a4eca1c85d` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:55.565641 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd8e7c-db54-4cc1-9621-680c65df1fdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd4a7bab198642d8b6ece9a4eca1c85d)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:55.567796 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd8e7c-db54-4cc1-9621-680c65df1fdb admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:55.623314 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-02fd8e7c-db54-4cc1-9621-680c65df1fdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd4a7bab198642d8b6ece9a4eca1c85d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:55.624020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-02fd8e7c-db54-4cc1-9621-680c65df1fdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd4a7bab198642d8b6ece9a4eca1c85d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:55.642368 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-02fd8e7c-db54-4cc1-9621-680c65df1fdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd4a7bab198642d8b6ece9a4eca1c85d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:55.656972 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-02fd8e7c-db54-4cc1-9621-680c65df1fdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd4a7bab198642d8b6ece9a4eca1c85d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:55.657722 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-02fd8e7c-db54-4cc1-9621-680c65df1fdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd4a7bab198642d8b6ece9a4eca1c85d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:55.672497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1330/2691] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:55 2022] DELETE /identity/v3/users/cd4a7bab198642d8b6ece9a4eca1c85d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:56.014712 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbf8e1-1fe5-4536-bbed-a3a6b4e182fa admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:56.016208 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0cfbf8e1-1fe5-4536-bbed-a3a6b4e182fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:56.029462 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbf8e1-1fe5-4536-bbed-a3a6b4e182fa admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:56.105060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b48669-b9c4-48ff-b208-aa047531300d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:56.105739 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b48669-b9c4-48ff-b208-aa047531300d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:56.106129 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b48669-b9c4-48ff-b208-aa047531300d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:56.106404 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b48669-b9c4-48ff-b208-aa047531300d admin admin] PATH_INFO: `/v3/projects/841086575f5f4a7c9ab412c093db25b4` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:56.107771 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b48669-b9c4-48ff-b208-aa047531300d admin admin] RBAC: Authorizing `identity:delete_project(project_id=841086575f5f4a7c9ab412c093db25b4)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:56.109605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b48669-b9c4-48ff-b208-aa047531300d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:56.122708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:56.169287 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-19b48669-b9c4-48ff-b208-aa047531300d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '841086575f5f4a7c9ab412c093db25b4', 'request_id': 'req-19b48669-b9c4-48ff-b208-aa047531300d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:56.175668 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-19b48669-b9c4-48ff-b208-aa047531300d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '841086575f5f4a7c9ab412c093db25b4', 'request_id': 'req-19b48669-b9c4-48ff-b208-aa047531300d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:56.182984 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-19b48669-b9c4-48ff-b208-aa047531300d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '841086575f5f4a7c9ab412c093db25b4', 'request_id': 'req-19b48669-b9c4-48ff-b208-aa047531300d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:56.197315 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-19b48669-b9c4-48ff-b208-aa047531300d admin admin] The token cache is being invalidate because project 841086575f5f4a7c9ab412c093db25b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:56.197639 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-19b48669-b9c4-48ff-b208-aa047531300d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:56.198318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-19b48669-b9c4-48ff-b208-aa047531300d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:56.199931 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1362/2692] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:56 2022] DELETE /identity/v3/projects/841086575f5f4a7c9ab412c093db25b4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:56.206021 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855d048-dd4e-4f77-9f7f-5c619c2afa32 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:56.206370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855d048-dd4e-4f77-9f7f-5c619c2afa32 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:56.206717 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855d048-dd4e-4f77-9f7f-5c619c2afa32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:56.233955 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-1855d048-dd4e-4f77-9f7f-5c619c2afa32 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:56.235907 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1855d048-dd4e-4f77-9f7f-5c619c2afa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:56.312222 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1331/2693] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:56.321638 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17370df6-3992-4a63-900e-292c5fa91718 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:56.322370 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17370df6-3992-4a63-900e-292c5fa91718 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:56.322599 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17370df6-3992-4a63-900e-292c5fa91718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:56.347535 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-17370df6-3992-4a63-900e-292c5fa91718 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:56.348759 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-17370df6-3992-4a63-900e-292c5fa91718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:56.357537 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1363/2694] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:56.362637 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855d048-dd4e-4f77-9f7f-5c619c2afa32 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:56.373356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855d048-dd4e-4f77-9f7f-5c619c2afa32 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:56.380092 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81320bff-e886-4b35-80fe-79b69ed7be02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:56.380790 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81320bff-e886-4b35-80fe-79b69ed7be02 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:56.381067 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81320bff-e886-4b35-80fe-79b69ed7be02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:56.381339 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81320bff-e886-4b35-80fe-79b69ed7be02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:56.382081 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81320bff-e886-4b35-80fe-79b69ed7be02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:09:56.382384 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81320bff-e886-4b35-80fe-79b69ed7be02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:56.384131 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81320bff-e886-4b35-80fe-79b69ed7be02 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:56.399984 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1332/2695] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:09:56.404523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17370df6-3992-4a63-900e-292c5fa91718 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:56.417435 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17370df6-3992-4a63-900e-292c5fa91718 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:56.424603 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ac157-1784-44cd-99fd-b5b583ba86ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:56.425362 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ac157-1784-44cd-99fd-b5b583ba86ce admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:56.425601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ac157-1784-44cd-99fd-b5b583ba86ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:56.425814 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ac157-1784-44cd-99fd-b5b583ba86ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:56.426524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197ac157-1784-44cd-99fd-b5b583ba86ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:56.428431 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197ac157-1784-44cd-99fd-b5b583ba86ce admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:56.435649 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81320bff-e886-4b35-80fe-79b69ed7be02 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:56.441634 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-81320bff-e886-4b35-80fe-79b69ed7be02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:56.460988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81320bff-e886-4b35-80fe-79b69ed7be02 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:56.467105 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c15ac-0d8a-4026-994b-fb1677f5fc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:56.468000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c15ac-0d8a-4026-994b-fb1677f5fc35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:56.468320 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c15ac-0d8a-4026-994b-fb1677f5fc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:56.468666 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c15ac-0d8a-4026-994b-fb1677f5fc35 admin admin] PATH_INFO: `/v3/users/1b4b80ab48bc4dab83fd03677544925c` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:56.471765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1364/2696] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:56 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:56.471957 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c15ac-0d8a-4026-994b-fb1677f5fc35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b4b80ab48bc4dab83fd03677544925c)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:56.475110 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c15ac-0d8a-4026-994b-fb1677f5fc35 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:56.477608 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ac157-1784-44cd-99fd-b5b583ba86ce admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:56.493585 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:56.544954 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-240c15ac-0d8a-4026-994b-fb1677f5fc35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b4b80ab48bc4dab83fd03677544925c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:56.545756 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-240c15ac-0d8a-4026-994b-fb1677f5fc35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b4b80ab48bc4dab83fd03677544925c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:56.556621 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-240c15ac-0d8a-4026-994b-fb1677f5fc35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b4b80ab48bc4dab83fd03677544925c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:56.574315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-240c15ac-0d8a-4026-994b-fb1677f5fc35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b4b80ab48bc4dab83fd03677544925c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:56.575681 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-240c15ac-0d8a-4026-994b-fb1677f5fc35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b4b80ab48bc4dab83fd03677544925c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:56.585820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ac157-1784-44cd-99fd-b5b583ba86ce admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:56.591329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e6179-be39-4128-9219-f7217bf2a2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:56.592115 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e6179-be39-4128-9219-f7217bf2a2fc admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:56.592861 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e6179-be39-4128-9219-f7217bf2a2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:56.593215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e6179-be39-4128-9219-f7217bf2a2fc admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:56.593929 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e6179-be39-4128-9219-f7217bf2a2fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:56.596917 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e6179-be39-4128-9219-f7217bf2a2fc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:56.599569 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1333/2697] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:56 2022] DELETE /identity/v3/users/1b4b80ab48bc4dab83fd03677544925c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:09:56.604651 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93967679-4867-44f5-b643-2c485c78dfb9 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:56.604938 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93967679-4867-44f5-b643-2c485c78dfb9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:56.605213 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93967679-4867-44f5-b643-2c485c78dfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:56.625939 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1365/2698] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:56 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:09:56.628171 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-93967679-4867-44f5-b643-2c485c78dfb9 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:56.629559 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-93967679-4867-44f5-b643-2c485c78dfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:56.630141 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e6179-be39-4128-9219-f7217bf2a2fc admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:56.631811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c69e6179-be39-4128-9219-f7217bf2a2fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:56.649133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e6179-be39-4128-9219-f7217bf2a2fc admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:56.692925 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1334/2699] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:09:56.704416 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93967679-4867-44f5-b643-2c485c78dfb9 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:56.706092 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-93967679-4867-44f5-b643-2c485c78dfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:56.713382 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb231b-200a-400d-9baa-abc36b3247a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:56.714048 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb231b-200a-400d-9baa-abc36b3247a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:56.714327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb231b-200a-400d-9baa-abc36b3247a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:56.714637 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb231b-200a-400d-9baa-abc36b3247a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:56.715290 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb231b-200a-400d-9baa-abc36b3247a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:56.717061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb231b-200a-400d-9baa-abc36b3247a2 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:56.719395 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:56.731343 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1366/2700] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:09:56.734955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb231b-200a-400d-9baa-abc36b3247a2 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:56.748369 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb231b-200a-400d-9baa-abc36b3247a2 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:56.756363 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9697b0cc-6a25-4eed-8463-1944a3342be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:56.757116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697b0cc-6a25-4eed-8463-1944a3342be9 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:56.757392 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697b0cc-6a25-4eed-8463-1944a3342be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:56.757663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697b0cc-6a25-4eed-8463-1944a3342be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:56.758627 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9697b0cc-6a25-4eed-8463-1944a3342be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:56.760432 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9697b0cc-6a25-4eed-8463-1944a3342be9 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:56.776549 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1367/2701] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:09:56.777802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9697b0cc-6a25-4eed-8463-1944a3342be9 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:56.788930 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93967679-4867-44f5-b643-2c485c78dfb9 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:56.792233 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9697b0cc-6a25-4eed-8463-1944a3342be9 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:56.793044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951207a-bbc7-4e5f-88e0-43019defca69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:56.793723 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951207a-bbc7-4e5f-88e0-43019defca69 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:56.794040 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951207a-bbc7-4e5f-88e0-43019defca69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:56.794316 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951207a-bbc7-4e5f-88e0-43019defca69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:56.797965 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73a461-fd3e-4f4f-8ed2-1a38c3e105fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:56.799451 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73a461-fd3e-4f4f-8ed2-1a38c3e105fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:56.799934 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73a461-fd3e-4f4f-8ed2-1a38c3e105fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:56.800395 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73a461-fd3e-4f4f-8ed2-1a38c3e105fd admin admin] PATH_INFO: `/v3/projects/2c620271a48c43d6a406399828a8ffaa/users/ffed99b36b95465397f625691891d0e3/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:56.805047 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951207a-bbc7-4e5f-88e0-43019defca69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:56.806530 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951207a-bbc7-4e5f-88e0-43019defca69 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:56.816524 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73a461-fd3e-4f4f-8ed2-1a38c3e105fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c620271a48c43d6a406399828a8ffaa, user_id=ffed99b36b95465397f625691891d0e3, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:56.817236 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1335/2702] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:09:56.820137 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73a461-fd3e-4f4f-8ed2-1a38c3e105fd admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:56.841584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1368/2703] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:56 2022] PUT /identity/v3/projects/2c620271a48c43d6a406399828a8ffaa/users/ffed99b36b95465397f625691891d0e3/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:56.854345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9b82c-f41f-4698-891d-c912af890351 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:56.854582 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9b82c-f41f-4698-891d-c912af890351 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:56.854867 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9b82c-f41f-4698-891d-c912af890351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:56.891178 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-36f9b82c-f41f-4698-891d-c912af890351 None None] MFA Rules not processed for user `ffed99b36b95465397f625691891d0e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:56.893244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-36f9b82c-f41f-4698-891d-c912af890351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:56.927162 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:56.992373 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1336/2704] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:56.998128 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73a461-fd3e-4f4f-8ed2-1a38c3e105fd admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:57.075989 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73a461-fd3e-4f4f-8ed2-1a38c3e105fd admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:57.082126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1638e7-08aa-45a2-946a-eda6a0364903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:57.082770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1638e7-08aa-45a2-946a-eda6a0364903 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:57.083055 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1638e7-08aa-45a2-946a-eda6a0364903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:57.083330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1638e7-08aa-45a2-946a-eda6a0364903 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:57.084001 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1638e7-08aa-45a2-946a-eda6a0364903 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:57.085953 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1638e7-08aa-45a2-946a-eda6a0364903 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:57.131754 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1369/2705] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:56 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:09:57.143119 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9b82c-f41f-4698-891d-c912af890351 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:57.143820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1638e7-08aa-45a2-946a-eda6a0364903 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:57.148927 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ec1638e7-08aa-45a2-946a-eda6a0364903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:57.160058 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9b82c-f41f-4698-891d-c912af890351 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:57.225174 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1638e7-08aa-45a2-946a-eda6a0364903 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:57.231105 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b036126-e42b-41fd-8959-021c57598dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:57.231766 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b036126-e42b-41fd-8959-021c57598dca admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:57.231969 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b25b99-4081-4a87-9ef0-99ecc621b37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:57.232109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b036126-e42b-41fd-8959-021c57598dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:57.232520 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b036126-e42b-41fd-8959-021c57598dca admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:57.232885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b25b99-4081-4a87-9ef0-99ecc621b37f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:57.233205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b036126-e42b-41fd-8959-021c57598dca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:57.233351 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b25b99-4081-4a87-9ef0-99ecc621b37f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:57.234018 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b25b99-4081-4a87-9ef0-99ecc621b37f admin admin] PATH_INFO: `/v3/projects/09b2794eca824cffae1b872da0beacea` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:57.236240 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b036126-e42b-41fd-8959-021c57598dca admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:57.236737 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b25b99-4081-4a87-9ef0-99ecc621b37f admin admin] RBAC: Authorizing `identity:delete_project(project_id=09b2794eca824cffae1b872da0beacea)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:57.240844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b25b99-4081-4a87-9ef0-99ecc621b37f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:57.264556 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1337/2706] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:57 2022] POST /identity/v3/users => generated 299 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:57.265870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b036126-e42b-41fd-8959-021c57598dca admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:57.278402 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b036126-e42b-41fd-8959-021c57598dca admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:57.284363 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a3797-12b4-4a05-8a37-79ccc04f27a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:57.285006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a3797-12b4-4a05-8a37-79ccc04f27a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:57.285333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a3797-12b4-4a05-8a37-79ccc04f27a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:57.285611 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a3797-12b4-4a05-8a37-79ccc04f27a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:57.287020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7a3797-12b4-4a05-8a37-79ccc04f27a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:57.288057 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7a3797-12b4-4a05-8a37-79ccc04f27a5 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:57.301392 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1338/2707] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:57.305000 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-36b25b99-4081-4a87-9ef0-99ecc621b37f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09b2794eca824cffae1b872da0beacea', 'request_id': 'req-36b25b99-4081-4a87-9ef0-99ecc621b37f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:57.309490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a3797-12b4-4a05-8a37-79ccc04f27a5 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:57.309720 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:57.313110 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-36b25b99-4081-4a87-9ef0-99ecc621b37f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09b2794eca824cffae1b872da0beacea', 'request_id': 'req-36b25b99-4081-4a87-9ef0-99ecc621b37f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:57.321180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a3797-12b4-4a05-8a37-79ccc04f27a5 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:57.322968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-36b25b99-4081-4a87-9ef0-99ecc621b37f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09b2794eca824cffae1b872da0beacea', 'request_id': 'req-36b25b99-4081-4a87-9ef0-99ecc621b37f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:57.340768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-36b25b99-4081-4a87-9ef0-99ecc621b37f admin admin] The token cache is being invalidate because project 09b2794eca824cffae1b872da0beacea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:57.341054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-36b25b99-4081-4a87-9ef0-99ecc621b37f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:57.341808 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-36b25b99-4081-4a87-9ef0-99ecc621b37f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:57.343626 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1370/2708] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:57 2022] DELETE /identity/v3/projects/09b2794eca824cffae1b872da0beacea => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:57.349220 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b25b99-4081-4a87-9ef0-99ecc621b37f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:57.352079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-36b25b99-4081-4a87-9ef0-99ecc621b37f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:57.406482 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd81f4-2b5c-4d8e-92af-071e4c8baeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:57.407083 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd81f4-2b5c-4d8e-92af-071e4c8baeec admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:57.407893 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd81f4-2b5c-4d8e-92af-071e4c8baeec admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:57.408353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd81f4-2b5c-4d8e-92af-071e4c8baeec admin admin] PATH_INFO: `/v3/projects/3eded3c409574caf8d770969cd481b2f/users/11934fd04408494f8d63c83baaa74225/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:57.429823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b25b99-4081-4a87-9ef0-99ecc621b37f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:57.430555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd81f4-2b5c-4d8e-92af-071e4c8baeec admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eded3c409574caf8d770969cd481b2f, user_id=11934fd04408494f8d63c83baaa74225, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:57.433261 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd81f4-2b5c-4d8e-92af-071e4c8baeec admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:57.434759 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac21c71-9d72-4544-a73a-a3d83cb1cf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:57.435607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac21c71-9d72-4544-a73a-a3d83cb1cf0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:57.435762 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac21c71-9d72-4544-a73a-a3d83cb1cf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:57.435995 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac21c71-9d72-4544-a73a-a3d83cb1cf0b admin admin] PATH_INFO: `/v3/users/fb8c73db9afc4d7d900e26e016ae1bba` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:57.437273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac21c71-9d72-4544-a73a-a3d83cb1cf0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb8c73db9afc4d7d900e26e016ae1bba)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:57.439230 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac21c71-9d72-4544-a73a-a3d83cb1cf0b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:57.451134 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1339/2709] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:57 2022] PUT /identity/v3/projects/3eded3c409574caf8d770969cd481b2f/users/11934fd04408494f8d63c83baaa74225/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:57.455208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd81f4-2b5c-4d8e-92af-071e4c8baeec admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:57.456969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-00cd81f4-2b5c-4d8e-92af-071e4c8baeec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:57.499084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:57.505006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dac21c71-9d72-4544-a73a-a3d83cb1cf0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb8c73db9afc4d7d900e26e016ae1bba'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:57.505689 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dac21c71-9d72-4544-a73a-a3d83cb1cf0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb8c73db9afc4d7d900e26e016ae1bba'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:57.515144 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dac21c71-9d72-4544-a73a-a3d83cb1cf0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb8c73db9afc4d7d900e26e016ae1bba'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:57.530071 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dac21c71-9d72-4544-a73a-a3d83cb1cf0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb8c73db9afc4d7d900e26e016ae1bba'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:57.540729 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dac21c71-9d72-4544-a73a-a3d83cb1cf0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb8c73db9afc4d7d900e26e016ae1bba'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:57.541489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd81f4-2b5c-4d8e-92af-071e4c8baeec admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:57.553772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1371/2710] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:57 2022] DELETE /identity/v3/users/fb8c73db9afc4d7d900e26e016ae1bba => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:57.554039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12d2da-2142-4228-8fd3-d6784907c5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:57.554039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12d2da-2142-4228-8fd3-d6784907c5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:57.554039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12d2da-2142-4228-8fd3-d6784907c5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:57.554039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12d2da-2142-4228-8fd3-d6784907c5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:57.554698 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12d2da-2142-4228-8fd3-d6784907c5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:57.555684 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12d2da-2142-4228-8fd3-d6784907c5d4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:57.569510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1340/2711] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:57.576965 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac21c71-9d72-4544-a73a-a3d83cb1cf0b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:57.578552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-dac21c71-9d72-4544-a73a-a3d83cb1cf0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:57.675383 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:57.679755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac21c71-9d72-4544-a73a-a3d83cb1cf0b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:57.685934 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ea422-0955-4376-80b7-faa9d58043d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:57.687094 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ea422-0955-4376-80b7-faa9d58043d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:57.690319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ea422-0955-4376-80b7-faa9d58043d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:57.690626 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ea422-0955-4376-80b7-faa9d58043d4 admin admin] PATH_INFO: `/v3/projects/3eded3c409574caf8d770969cd481b2f/users/11934fd04408494f8d63c83baaa74225/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:57.692932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ea422-0955-4376-80b7-faa9d58043d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eded3c409574caf8d770969cd481b2f, user_id=11934fd04408494f8d63c83baaa74225, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:57.695633 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ea422-0955-4376-80b7-faa9d58043d4 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:57.709356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1372/2712] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:57 2022] PUT /identity/v3/projects/3eded3c409574caf8d770969cd481b2f/users/11934fd04408494f8d63c83baaa74225/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:57.720708 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645ff13-e4fd-4aae-a38d-d82d6c7adcc7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:57.721292 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645ff13-e4fd-4aae-a38d-d82d6c7adcc7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:57.721292 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645ff13-e4fd-4aae-a38d-d82d6c7adcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:57.761618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4645ff13-e4fd-4aae-a38d-d82d6c7adcc7 None None] MFA Rules not processed for user `11934fd04408494f8d63c83baaa74225`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:57.762912 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4645ff13-e4fd-4aae-a38d-d82d6c7adcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:57.849250 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1341/2713] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:57.859255 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ea422-0955-4376-80b7-faa9d58043d4 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:57.861539 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d67ea422-0955-4376-80b7-faa9d58043d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:57.905089 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645ff13-e4fd-4aae-a38d-d82d6c7adcc7 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:57.908159 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4645ff13-e4fd-4aae-a38d-d82d6c7adcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:57.946864 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ea422-0955-4376-80b7-faa9d58043d4 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:57.950988 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae891165-e813-4c10-8655-737f3f80b2e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:57.951623 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae891165-e813-4c10-8655-737f3f80b2e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:57.951910 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae891165-e813-4c10-8655-737f3f80b2e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:57.952185 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae891165-e813-4c10-8655-737f3f80b2e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:57.963170 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae891165-e813-4c10-8655-737f3f80b2e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:57.964636 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae891165-e813-4c10-8655-737f3f80b2e4 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:57.976491 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1373/2714] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:58.010320 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645ff13-e4fd-4aae-a38d-d82d6c7adcc7 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:58.015293 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47b2b2-b627-4861-82fa-6373e5db6445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:58.015947 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47b2b2-b627-4861-82fa-6373e5db6445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:58.016231 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47b2b2-b627-4861-82fa-6373e5db6445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:58.016504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47b2b2-b627-4861-82fa-6373e5db6445 admin admin] PATH_INFO: `/v3/projects/b648ab031a90453c935145efba14ee56` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:58.018307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab47b2b2-b627-4861-82fa-6373e5db6445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b648ab031a90453c935145efba14ee56)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:58.020230 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab47b2b2-b627-4861-82fa-6373e5db6445 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:58.103264 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ab47b2b2-b627-4861-82fa-6373e5db6445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b648ab031a90453c935145efba14ee56', 'request_id': 'req-ab47b2b2-b627-4861-82fa-6373e5db6445'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:58.105525 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:58.112604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ab47b2b2-b627-4861-82fa-6373e5db6445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b648ab031a90453c935145efba14ee56', 'request_id': 'req-ab47b2b2-b627-4861-82fa-6373e5db6445'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:58.118274 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ab47b2b2-b627-4861-82fa-6373e5db6445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b648ab031a90453c935145efba14ee56', 'request_id': 'req-ab47b2b2-b627-4861-82fa-6373e5db6445'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:58.137215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ab47b2b2-b627-4861-82fa-6373e5db6445 admin admin] The token cache is being invalidate because project b648ab031a90453c935145efba14ee56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:58.137508 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ab47b2b2-b627-4861-82fa-6373e5db6445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:58.138324 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ab47b2b2-b627-4861-82fa-6373e5db6445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:58.139876 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1342/2715] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:57 2022] DELETE /identity/v3/projects/b648ab031a90453c935145efba14ee56 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:58.147976 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9022e-1219-4cd5-8c6c-359a6279c5ee None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:58.148264 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9022e-1219-4cd5-8c6c-359a6279c5ee None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:58.148541 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9022e-1219-4cd5-8c6c-359a6279c5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:58.173821 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1bb9022e-1219-4cd5-8c6c-359a6279c5ee None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:58.175244 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1bb9022e-1219-4cd5-8c6c-359a6279c5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:58.257739 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1374/2716] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:58.267845 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02924c1-e0cd-4670-9f55-e3706f3fe287 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:58.268231 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02924c1-e0cd-4670-9f55-e3706f3fe287 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:58.268573 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02924c1-e0cd-4670-9f55-e3706f3fe287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:58.289170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c02924c1-e0cd-4670-9f55-e3706f3fe287 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:58.290639 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c02924c1-e0cd-4670-9f55-e3706f3fe287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:58.300204 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1343/2717] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:09:58.311872 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9022e-1219-4cd5-8c6c-359a6279c5ee None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:58.316856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9022e-1219-4cd5-8c6c-359a6279c5ee None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:58.322262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3774d1c-a9f5-4026-b45e-413a9a27a614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:58.322894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3774d1c-a9f5-4026-b45e-413a9a27a614 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:58.323180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3774d1c-a9f5-4026-b45e-413a9a27a614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:58.323757 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3774d1c-a9f5-4026-b45e-413a9a27a614 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:58.324142 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3774d1c-a9f5-4026-b45e-413a9a27a614 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:09:58.324484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3774d1c-a9f5-4026-b45e-413a9a27a614 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:58.326257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3774d1c-a9f5-4026-b45e-413a9a27a614 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:58.340238 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1375/2718] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:09:58.345041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02924c1-e0cd-4670-9f55-e3706f3fe287 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:58.355342 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02924c1-e0cd-4670-9f55-e3706f3fe287 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:58.360248 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a6ccd-b493-4f2f-9039-bfaeb1885bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:58.361423 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a6ccd-b493-4f2f-9039-bfaeb1885bbb admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:58.361706 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a6ccd-b493-4f2f-9039-bfaeb1885bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:58.362024 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a6ccd-b493-4f2f-9039-bfaeb1885bbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:58.362697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a6ccd-b493-4f2f-9039-bfaeb1885bbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:58.364884 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a6ccd-b493-4f2f-9039-bfaeb1885bbb admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:58.396917 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1344/2719] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:09:58 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:09:58.401464 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3774d1c-a9f5-4026-b45e-413a9a27a614 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:58.423895 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3774d1c-a9f5-4026-b45e-413a9a27a614 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:58.500758 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ba455-ed78-4e82-93e7-4ba0d3db4192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:58.501959 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ba455-ed78-4e82-93e7-4ba0d3db4192 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:58.502237 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ba455-ed78-4e82-93e7-4ba0d3db4192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:58.502502 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ba455-ed78-4e82-93e7-4ba0d3db4192 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:58.503182 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ba455-ed78-4e82-93e7-4ba0d3db4192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:58.505129 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ba455-ed78-4e82-93e7-4ba0d3db4192 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:58.528738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1376/2720] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:09:58 2022] POST /identity/v3/users => generated 287 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:09:58.535050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a6ccd-b493-4f2f-9039-bfaeb1885bbb admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:58.546171 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a6ccd-b493-4f2f-9039-bfaeb1885bbb admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:58.553041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02915be3-8855-452b-9e00-35e9ba59d05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:58.553674 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02915be3-8855-452b-9e00-35e9ba59d05f admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:58.553992 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02915be3-8855-452b-9e00-35e9ba59d05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:58.554287 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02915be3-8855-452b-9e00-35e9ba59d05f admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:58.554978 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02915be3-8855-452b-9e00-35e9ba59d05f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:58.556601 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02915be3-8855-452b-9e00-35e9ba59d05f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:58.569575 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1345/2721] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:58.574513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ba455-ed78-4e82-93e7-4ba0d3db4192 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:58.587906 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ba455-ed78-4e82-93e7-4ba0d3db4192 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:58.594301 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c388365-7e66-44dd-ba66-54b36c4efeaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:58.594943 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c388365-7e66-44dd-ba66-54b36c4efeaa admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:58.595223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c388365-7e66-44dd-ba66-54b36c4efeaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:58.595601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c388365-7e66-44dd-ba66-54b36c4efeaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:58.596679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c388365-7e66-44dd-ba66-54b36c4efeaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:58.599720 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c388365-7e66-44dd-ba66-54b36c4efeaa admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:58.611783 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:58.621799 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1377/2722] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:09:58.625913 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02915be3-8855-452b-9e00-35e9ba59d05f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:58.639753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02915be3-8855-452b-9e00-35e9ba59d05f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:58.645208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e1ff1-93e9-446e-8cd2-332b38080b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:58.645908 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e1ff1-93e9-446e-8cd2-332b38080b83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:58.646233 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e1ff1-93e9-446e-8cd2-332b38080b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:58.646720 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e1ff1-93e9-446e-8cd2-332b38080b83 admin admin] PATH_INFO: `/v3/projects/53f9adda77ff445b8eecd09fa7184530/users/1026208ad63143f589f05818ce082174/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:58.664213 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c388365-7e66-44dd-ba66-54b36c4efeaa admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:58.665154 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e1ff1-93e9-446e-8cd2-332b38080b83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53f9adda77ff445b8eecd09fa7184530, user_id=1026208ad63143f589f05818ce082174, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:58.666226 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0c388365-7e66-44dd-ba66-54b36c4efeaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:58.668140 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e1ff1-93e9-446e-8cd2-332b38080b83 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:58.679577 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c388365-7e66-44dd-ba66-54b36c4efeaa admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:58.682483 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1346/2723] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:09:58 2022] PUT /identity/v3/projects/53f9adda77ff445b8eecd09fa7184530/users/1026208ad63143f589f05818ce082174/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:58.693127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c0fd1-9f4a-4700-83b2-c93e8e5d16ff None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:58.693353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c0fd1-9f4a-4700-83b2-c93e8e5d16ff None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:58.693628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c0fd1-9f4a-4700-83b2-c93e8e5d16ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:58.723573 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-267c0fd1-9f4a-4700-83b2-c93e8e5d16ff None None] MFA Rules not processed for user `1026208ad63143f589f05818ce082174`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:58.724791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-267c0fd1-9f4a-4700-83b2-c93e8e5d16ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:58.746340 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea187c-dabf-4426-af23-0120c249f9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:58.747418 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea187c-dabf-4426-af23-0120c249f9b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:58.747940 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea187c-dabf-4426-af23-0120c249f9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:58.748417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea187c-dabf-4426-af23-0120c249f9b1 admin admin] PATH_INFO: `/v3/users/ffed99b36b95465397f625691891d0e3` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:58.750453 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea187c-dabf-4426-af23-0120c249f9b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffed99b36b95465397f625691891d0e3)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:58.754329 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea187c-dabf-4426-af23-0120c249f9b1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:58.799848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1347/2724] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:09:58.811331 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c0fd1-9f4a-4700-83b2-c93e8e5d16ff None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:58.812853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-267c0fd1-9f4a-4700-83b2-c93e8e5d16ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:58.817652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-87ea187c-dabf-4426-af23-0120c249f9b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffed99b36b95465397f625691891d0e3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:58.818368 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-87ea187c-dabf-4426-af23-0120c249f9b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffed99b36b95465397f625691891d0e3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:58.827366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-87ea187c-dabf-4426-af23-0120c249f9b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffed99b36b95465397f625691891d0e3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:58.844202 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-87ea187c-dabf-4426-af23-0120c249f9b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffed99b36b95465397f625691891d0e3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:58.852554 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-87ea187c-dabf-4426-af23-0120c249f9b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffed99b36b95465397f625691891d0e3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:58.863996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1378/2725] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:58 2022] DELETE /identity/v3/users/ffed99b36b95465397f625691891d0e3 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:58.865733 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27fe29-c622-4ec0-b70f-5b4445e2f07f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:58.866059 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27fe29-c622-4ec0-b70f-5b4445e2f07f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:58.866339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27fe29-c622-4ec0-b70f-5b4445e2f07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:58.879888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c0fd1-9f4a-4700-83b2-c93e8e5d16ff None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:58.886525 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1e27fe29-c622-4ec0-b70f-5b4445e2f07f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:09:58.887748 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1e27fe29-c622-4ec0-b70f-5b4445e2f07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:58.898911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:58.936341 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09103674-8d6a-4c78-8b74-b0a9cd724676 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:58.936976 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09103674-8d6a-4c78-8b74-b0a9cd724676 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:58.937263 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09103674-8d6a-4c78-8b74-b0a9cd724676 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:58.937532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09103674-8d6a-4c78-8b74-b0a9cd724676 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:58.939431 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-09103674-8d6a-4c78-8b74-b0a9cd724676 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:58.960054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1379/2726] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:09:58.971617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27fe29-c622-4ec0-b70f-5b4445e2f07f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:58.976360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1e27fe29-c622-4ec0-b70f-5b4445e2f07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:59.009119 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:09:59.020845 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09103674-8d6a-4c78-8b74-b0a9cd724676 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:59.022359 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09103674-8d6a-4c78-8b74-b0a9cd724676 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:59.033498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1348/2727] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:09:59.064993 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27fe29-c622-4ec0-b70f-5b4445e2f07f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:59.069906 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64492fd-4d01-49d3-94a5-0709162f4159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:59.070550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64492fd-4d01-49d3-94a5-0709162f4159 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:59.070827 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64492fd-4d01-49d3-94a5-0709162f4159 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:59.071093 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64492fd-4d01-49d3-94a5-0709162f4159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:59.082164 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64492fd-4d01-49d3-94a5-0709162f4159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:59.083617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64492fd-4d01-49d3-94a5-0709162f4159 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:59.094496 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1380/2728] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:09:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:09:59.428073 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09103674-8d6a-4c78-8b74-b0a9cd724676 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:59.430085 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-09103674-8d6a-4c78-8b74-b0a9cd724676 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:09:59.452215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09103674-8d6a-4c78-8b74-b0a9cd724676 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:59.461513 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608763c5-0866-4b3c-80b5-d2a00bbb3945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:59.463619 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608763c5-0866-4b3c-80b5-d2a00bbb3945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:59.464287 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608763c5-0866-4b3c-80b5-d2a00bbb3945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:59.464743 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608763c5-0866-4b3c-80b5-d2a00bbb3945 admin admin] PATH_INFO: `/v3/projects/2c620271a48c43d6a406399828a8ffaa` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:59.467830 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608763c5-0866-4b3c-80b5-d2a00bbb3945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c620271a48c43d6a406399828a8ffaa)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:59.472217 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608763c5-0866-4b3c-80b5-d2a00bbb3945 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:59.533306 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-608763c5-0866-4b3c-80b5-d2a00bbb3945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c620271a48c43d6a406399828a8ffaa', 'request_id': 'req-608763c5-0866-4b3c-80b5-d2a00bbb3945'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:59.541250 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-608763c5-0866-4b3c-80b5-d2a00bbb3945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c620271a48c43d6a406399828a8ffaa', 'request_id': 'req-608763c5-0866-4b3c-80b5-d2a00bbb3945'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:59.547284 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-608763c5-0866-4b3c-80b5-d2a00bbb3945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c620271a48c43d6a406399828a8ffaa', 'request_id': 'req-608763c5-0866-4b3c-80b5-d2a00bbb3945'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:59.560304 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-608763c5-0866-4b3c-80b5-d2a00bbb3945 admin admin] The token cache is being invalidate because project 2c620271a48c43d6a406399828a8ffaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:09:59.560567 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-608763c5-0866-4b3c-80b5-d2a00bbb3945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:59.561210 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-608763c5-0866-4b3c-80b5-d2a00bbb3945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:59.562761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1349/2729] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:09:59 2022] DELETE /identity/v3/projects/2c620271a48c43d6a406399828a8ffaa => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:09:59.567314 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64492fd-4d01-49d3-94a5-0709162f4159 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:09:59.577791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64492fd-4d01-49d3-94a5-0709162f4159 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:09:59.642642 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f698dd4-4878-41b4-91f9-30102d49b911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:09:59.643290 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f698dd4-4878-41b4-91f9-30102d49b911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:09:59.643572 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f698dd4-4878-41b4-91f9-30102d49b911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:09:59.643851 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f698dd4-4878-41b4-91f9-30102d49b911 admin admin] PATH_INFO: `/v3/users/11934fd04408494f8d63c83baaa74225` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:09:59.645140 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f698dd4-4878-41b4-91f9-30102d49b911 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11934fd04408494f8d63c83baaa74225)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:09:59.647113 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f698dd4-4878-41b4-91f9-30102d49b911 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:09:59.701118 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7f698dd4-4878-41b4-91f9-30102d49b911 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11934fd04408494f8d63c83baaa74225'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:59.701802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7f698dd4-4878-41b4-91f9-30102d49b911 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11934fd04408494f8d63c83baaa74225'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:59.710171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7f698dd4-4878-41b4-91f9-30102d49b911 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11934fd04408494f8d63c83baaa74225'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:59.723171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7f698dd4-4878-41b4-91f9-30102d49b911 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11934fd04408494f8d63c83baaa74225'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:59.730535 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7f698dd4-4878-41b4-91f9-30102d49b911 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11934fd04408494f8d63c83baaa74225'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:09:59.736876 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1381/2730] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:09:59 2022] DELETE /identity/v3/users/11934fd04408494f8d63c83baaa74225 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:10:00.050261 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608763c5-0866-4b3c-80b5-d2a00bbb3945 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:00.052102 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-608763c5-0866-4b3c-80b5-d2a00bbb3945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:00.082531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608763c5-0866-4b3c-80b5-d2a00bbb3945 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:00.152178 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392b107-456d-4518-8dae-ec944529dd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:00.152838 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392b107-456d-4518-8dae-ec944529dd45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:00.153125 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392b107-456d-4518-8dae-ec944529dd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:00.153403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392b107-456d-4518-8dae-ec944529dd45 admin admin] PATH_INFO: `/v3/projects/3eded3c409574caf8d770969cd481b2f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:00.154606 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392b107-456d-4518-8dae-ec944529dd45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eded3c409574caf8d770969cd481b2f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:00.156528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392b107-456d-4518-8dae-ec944529dd45 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:00.158298 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:00.210642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a392b107-456d-4518-8dae-ec944529dd45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eded3c409574caf8d770969cd481b2f', 'request_id': 'req-a392b107-456d-4518-8dae-ec944529dd45'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:00.218315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a392b107-456d-4518-8dae-ec944529dd45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eded3c409574caf8d770969cd481b2f', 'request_id': 'req-a392b107-456d-4518-8dae-ec944529dd45'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:00.225587 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a392b107-456d-4518-8dae-ec944529dd45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eded3c409574caf8d770969cd481b2f', 'request_id': 'req-a392b107-456d-4518-8dae-ec944529dd45'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:00.242336 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a392b107-456d-4518-8dae-ec944529dd45 admin admin] The token cache is being invalidate because project 3eded3c409574caf8d770969cd481b2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:10:00.242624 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a392b107-456d-4518-8dae-ec944529dd45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:00.243182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a392b107-456d-4518-8dae-ec944529dd45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:00.244728 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1350/2731] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:10:00 2022] DELETE /identity/v3/projects/3eded3c409574caf8d770969cd481b2f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:10:00.251192 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cde125-a705-4a64-a06b-b4d20f54f43f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:00.251722 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cde125-a705-4a64-a06b-b4d20f54f43f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:00.252217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cde125-a705-4a64-a06b-b4d20f54f43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:00.277047 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-44cde125-a705-4a64-a06b-b4d20f54f43f None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:00.278299 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-44cde125-a705-4a64-a06b-b4d20f54f43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:00.320753 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:00.354961 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1382/2732] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:10:00.364889 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b68a16-d029-45f4-b483-8cb738a28b7c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:00.365365 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b68a16-d029-45f4-b483-8cb738a28b7c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:00.365738 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b68a16-d029-45f4-b483-8cb738a28b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:00.386534 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-26b68a16-d029-45f4-b483-8cb738a28b7c None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:00.387752 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-26b68a16-d029-45f4-b483-8cb738a28b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:00.395742 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1351/2733] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:10:00.400205 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cde125-a705-4a64-a06b-b4d20f54f43f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:00.411658 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cde125-a705-4a64-a06b-b4d20f54f43f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:00.416417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c358b5-8816-4ab0-b444-de0e4ce6d9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:00.417099 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c358b5-8816-4ab0-b444-de0e4ce6d9ee admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:00.417382 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c358b5-8816-4ab0-b444-de0e4ce6d9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:00.417648 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c358b5-8816-4ab0-b444-de0e4ce6d9ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:00.418386 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c358b5-8816-4ab0-b444-de0e4ce6d9ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:10:00.418682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c358b5-8816-4ab0-b444-de0e4ce6d9ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:00.420282 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c358b5-8816-4ab0-b444-de0e4ce6d9ee admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:00.433558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1383/2734] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:10:00.437999 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b68a16-d029-45f4-b483-8cb738a28b7c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:00.449873 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b68a16-d029-45f4-b483-8cb738a28b7c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:00.455137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60505fbf-f1d2-418f-8a6a-12d349ea95c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:00.455770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60505fbf-f1d2-418f-8a6a-12d349ea95c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:00.456048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60505fbf-f1d2-418f-8a6a-12d349ea95c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:00.456324 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60505fbf-f1d2-418f-8a6a-12d349ea95c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:00.456993 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60505fbf-f1d2-418f-8a6a-12d349ea95c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:00.460271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60505fbf-f1d2-418f-8a6a-12d349ea95c7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:00.468124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79791310-5de5-492e-8479-0b69398e603e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:00.468550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79791310-5de5-492e-8479-0b69398e603e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:00.468869 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79791310-5de5-492e-8479-0b69398e603e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:00.489718 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-79791310-5de5-492e-8479-0b69398e603e None None] MFA Rules not processed for user `1026208ad63143f589f05818ce082174`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:00.491564 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-79791310-5de5-492e-8479-0b69398e603e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:00.499814 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1352/2735] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:10:00 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:10:00.500957 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60505fbf-f1d2-418f-8a6a-12d349ea95c7 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:00.573767 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1384/2736] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:10:00.589428 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60505fbf-f1d2-418f-8a6a-12d349ea95c7 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:00.594184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ea004-eaba-4797-a226-ed3047197655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:00.595417 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ea004-eaba-4797-a226-ed3047197655 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:00.595712 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ea004-eaba-4797-a226-ed3047197655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:00.596003 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ea004-eaba-4797-a226-ed3047197655 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:00.596670 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ea004-eaba-4797-a226-ed3047197655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:00.598635 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ea004-eaba-4797-a226-ed3047197655 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:00.622672 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1353/2737] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:10:00 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:10:00.626973 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79791310-5de5-492e-8479-0b69398e603e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:00.638888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79791310-5de5-492e-8479-0b69398e603e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:00.644898 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7804f-98eb-4fc5-abf9-274a35e4ca10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:00.645518 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7804f-98eb-4fc5-abf9-274a35e4ca10 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:00.645799 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7804f-98eb-4fc5-abf9-274a35e4ca10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:00.646114 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7804f-98eb-4fc5-abf9-274a35e4ca10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:00.646821 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7804f-98eb-4fc5-abf9-274a35e4ca10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:00.648448 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7804f-98eb-4fc5-abf9-274a35e4ca10 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:00.660160 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1385/2738] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:10:00.664403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ea004-eaba-4797-a226-ed3047197655 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:00.675063 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ea004-eaba-4797-a226-ed3047197655 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:00.679845 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a84c1-ebaf-42cc-a17a-5d13ca2b3dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:00.680490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a84c1-ebaf-42cc-a17a-5d13ca2b3dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:00.681130 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a84c1-ebaf-42cc-a17a-5d13ca2b3dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:00.681130 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a84c1-ebaf-42cc-a17a-5d13ca2b3dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:00.681744 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a84c1-ebaf-42cc-a17a-5d13ca2b3dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:00.683441 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a84c1-ebaf-42cc-a17a-5d13ca2b3dc9 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:00.694729 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1354/2739] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:10:00.698971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7804f-98eb-4fc5-abf9-274a35e4ca10 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:00.717852 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7804f-98eb-4fc5-abf9-274a35e4ca10 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:00.725127 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f1ab4-d1d7-4564-8476-60c2fd48418f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:00.726992 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f1ab4-d1d7-4564-8476-60c2fd48418f admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:00.726992 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f1ab4-d1d7-4564-8476-60c2fd48418f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:00.726992 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f1ab4-d1d7-4564-8476-60c2fd48418f admin admin] PATH_INFO: `/v3/projects/ccf93459e069439a82f942a4ca5b0f4c/users/c08644623aa64570a8d64ec2b0e2e01c/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:00.760444 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f1ab4-d1d7-4564-8476-60c2fd48418f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf93459e069439a82f942a4ca5b0f4c, user_id=c08644623aa64570a8d64ec2b0e2e01c, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:00.763965 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f1ab4-d1d7-4564-8476-60c2fd48418f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:00.779640 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1386/2740] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:10:00 2022] PUT /identity/v3/projects/ccf93459e069439a82f942a4ca5b0f4c/users/c08644623aa64570a8d64ec2b0e2e01c/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:10:00.792033 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173abc12-72f1-493c-8ff4-dae40e6c0bd1 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:00.792418 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173abc12-72f1-493c-8ff4-dae40e6c0bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:00.792775 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173abc12-72f1-493c-8ff4-dae40e6c0bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:00.829936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-173abc12-72f1-493c-8ff4-dae40e6c0bd1 None None] MFA Rules not processed for user `c08644623aa64570a8d64ec2b0e2e01c`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:00.831273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-173abc12-72f1-493c-8ff4-dae40e6c0bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:00.863997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:00.903123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1355/2741] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:10:00.918998 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f1ab4-d1d7-4564-8476-60c2fd48418f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:00.920878 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-374f1ab4-d1d7-4564-8476-60c2fd48418f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:00.996244 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f1ab4-d1d7-4564-8476-60c2fd48418f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:01.001604 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453334d5-5f07-4f28-a65e-1cc51e368911 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:01.002280 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453334d5-5f07-4f28-a65e-1cc51e368911 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:01.002560 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453334d5-5f07-4f28-a65e-1cc51e368911 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:01.002838 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453334d5-5f07-4f28-a65e-1cc51e368911 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:01.012999 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453334d5-5f07-4f28-a65e-1cc51e368911 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:01.014404 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453334d5-5f07-4f28-a65e-1cc51e368911 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:01.024877 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1387/2742] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:10:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:10:01.045160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173abc12-72f1-493c-8ff4-dae40e6c0bd1 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:01.107708 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453334d5-5f07-4f28-a65e-1cc51e368911 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:01.109366 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-453334d5-5f07-4f28-a65e-1cc51e368911 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:01.137476 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173abc12-72f1-493c-8ff4-dae40e6c0bd1 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:01.146356 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6223709-ccfb-4c46-b83c-44b044084a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:01.147997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6223709-ccfb-4c46-b83c-44b044084a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:01.148274 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6223709-ccfb-4c46-b83c-44b044084a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:01.148545 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6223709-ccfb-4c46-b83c-44b044084a15 admin admin] PATH_INFO: `/v3/users/c08644623aa64570a8d64ec2b0e2e01c` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:01.150080 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6223709-ccfb-4c46-b83c-44b044084a15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c08644623aa64570a8d64ec2b0e2e01c)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:01.152032 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6223709-ccfb-4c46-b83c-44b044084a15 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:01.165562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:01.201378 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453334d5-5f07-4f28-a65e-1cc51e368911 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:01.207844 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d7a66-5371-4246-9f7e-bffc1f393a93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:01.208539 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d7a66-5371-4246-9f7e-bffc1f393a93 service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:01.208829 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d7a66-5371-4246-9f7e-bffc1f393a93 service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:01.209102 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d7a66-5371-4246-9f7e-bffc1f393a93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:01.211180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f75d7a66-5371-4246-9f7e-bffc1f393a93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:01.214075 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f6223709-ccfb-4c46-b83c-44b044084a15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c08644623aa64570a8d64ec2b0e2e01c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:01.214770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f6223709-ccfb-4c46-b83c-44b044084a15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c08644623aa64570a8d64ec2b0e2e01c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:01.224383 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f6223709-ccfb-4c46-b83c-44b044084a15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c08644623aa64570a8d64ec2b0e2e01c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:01.226779 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d7a66-5371-4246-9f7e-bffc1f393a93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:01.228442 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d7a66-5371-4246-9f7e-bffc1f393a93 service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:01.239435 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f6223709-ccfb-4c46-b83c-44b044084a15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c08644623aa64570a8d64ec2b0e2e01c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:01.239732 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1388/2743] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:10:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:10:01.240631 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f6223709-ccfb-4c46-b83c-44b044084a15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c08644623aa64570a8d64ec2b0e2e01c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:01.253845 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1356/2744] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:10:01 2022] DELETE /identity/v3/users/c08644623aa64570a8d64ec2b0e2e01c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:10:01.258425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28779e-29c1-46fc-9256-7a48f5d0da21 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:01.258704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28779e-29c1-46fc-9256-7a48f5d0da21 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:01.258980 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28779e-29c1-46fc-9256-7a48f5d0da21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:01.287199 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-0f28779e-29c1-46fc-9256-7a48f5d0da21 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:01.288643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0f28779e-29c1-46fc-9256-7a48f5d0da21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:01.371177 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1389/2745] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:10:01.381477 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6223709-ccfb-4c46-b83c-44b044084a15 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:01.382991 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f6223709-ccfb-4c46-b83c-44b044084a15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:01.444323 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:01.449997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6223709-ccfb-4c46-b83c-44b044084a15 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:01.455527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87440e2f-6ba3-4821-90e3-ec35ecb7c860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:01.456608 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87440e2f-6ba3-4821-90e3-ec35ecb7c860 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:01.457123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87440e2f-6ba3-4821-90e3-ec35ecb7c860 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:01.457745 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87440e2f-6ba3-4821-90e3-ec35ecb7c860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:01.468789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87440e2f-6ba3-4821-90e3-ec35ecb7c860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:01.470359 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87440e2f-6ba3-4821-90e3-ec35ecb7c860 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:01.484353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1357/2746] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:10:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:10:01.822031 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28779e-29c1-46fc-9256-7a48f5d0da21 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:01.824550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0f28779e-29c1-46fc-9256-7a48f5d0da21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:01.837638 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28779e-29c1-46fc-9256-7a48f5d0da21 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:01.844319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d092e5-6c18-4e9b-9dc8-a666dfacd561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:01.846190 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d092e5-6c18-4e9b-9dc8-a666dfacd561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:01.846806 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d092e5-6c18-4e9b-9dc8-a666dfacd561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:01.847052 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d092e5-6c18-4e9b-9dc8-a666dfacd561 admin admin] PATH_INFO: `/v3/projects/ccf93459e069439a82f942a4ca5b0f4c` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:01.848581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d092e5-6c18-4e9b-9dc8-a666dfacd561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccf93459e069439a82f942a4ca5b0f4c)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:01.850754 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d092e5-6c18-4e9b-9dc8-a666dfacd561 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:01.907080 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f8d092e5-6c18-4e9b-9dc8-a666dfacd561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccf93459e069439a82f942a4ca5b0f4c', 'request_id': 'req-f8d092e5-6c18-4e9b-9dc8-a666dfacd561'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:01.918204 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f8d092e5-6c18-4e9b-9dc8-a666dfacd561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccf93459e069439a82f942a4ca5b0f4c', 'request_id': 'req-f8d092e5-6c18-4e9b-9dc8-a666dfacd561'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:01.927477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f8d092e5-6c18-4e9b-9dc8-a666dfacd561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccf93459e069439a82f942a4ca5b0f4c', 'request_id': 'req-f8d092e5-6c18-4e9b-9dc8-a666dfacd561'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:01.941127 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f8d092e5-6c18-4e9b-9dc8-a666dfacd561 admin admin] The token cache is being invalidate because project ccf93459e069439a82f942a4ca5b0f4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:10:01.941543 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f8d092e5-6c18-4e9b-9dc8-a666dfacd561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:01.942267 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f8d092e5-6c18-4e9b-9dc8-a666dfacd561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:01.943856 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1390/2747] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:10:01 2022] DELETE /identity/v3/projects/ccf93459e069439a82f942a4ca5b0f4c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:10:01.949081 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d8a66-3517-46ad-8666-492eb5e7c699 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:01.949547 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d8a66-3517-46ad-8666-492eb5e7c699 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:01.949999 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d8a66-3517-46ad-8666-492eb5e7c699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:01.974339 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e97d8a66-3517-46ad-8666-492eb5e7c699 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:01.975594 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e97d8a66-3517-46ad-8666-492eb5e7c699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:02.043993 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1358/2748] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:10:02.056898 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357aa8a8-e119-4f2c-9874-e6eb90fd0f2b None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:02.056898 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357aa8a8-e119-4f2c-9874-e6eb90fd0f2b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:02.056898 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357aa8a8-e119-4f2c-9874-e6eb90fd0f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:02.078083 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-357aa8a8-e119-4f2c-9874-e6eb90fd0f2b None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:02.079250 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-357aa8a8-e119-4f2c-9874-e6eb90fd0f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:02.088840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1391/2749] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:10:02.093189 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d8a66-3517-46ad-8666-492eb5e7c699 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:02.104219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d8a66-3517-46ad-8666-492eb5e7c699 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:02.109349 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbd0aa-3593-4b31-8596-b07d0c9caab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:02.110076 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbd0aa-3593-4b31-8596-b07d0c9caab7 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:02.110363 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbd0aa-3593-4b31-8596-b07d0c9caab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:02.111215 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbd0aa-3593-4b31-8596-b07d0c9caab7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:02.111983 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbbd0aa-3593-4b31-8596-b07d0c9caab7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:10:02.112291 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbbd0aa-3593-4b31-8596-b07d0c9caab7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:02.114340 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbbd0aa-3593-4b31-8596-b07d0c9caab7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:02.128521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1359/2750] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:10:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:10:02.132909 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357aa8a8-e119-4f2c-9874-e6eb90fd0f2b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:02.146342 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357aa8a8-e119-4f2c-9874-e6eb90fd0f2b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:02.151751 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd1460-6dd3-4851-97c8-9b97815e3d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:02.152393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd1460-6dd3-4851-97c8-9b97815e3d22 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:02.152678 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd1460-6dd3-4851-97c8-9b97815e3d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:02.152949 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd1460-6dd3-4851-97c8-9b97815e3d22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:02.153692 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd1460-6dd3-4851-97c8-9b97815e3d22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:02.155687 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd1460-6dd3-4851-97c8-9b97815e3d22 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:02.185582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1392/2751] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:10:02 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:10:02.190036 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbd0aa-3593-4b31-8596-b07d0c9caab7 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:02.200619 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbd0aa-3593-4b31-8596-b07d0c9caab7 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:02.269407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c611ff62-9e13-4679-aebe-247092f1d740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:02.270386 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611ff62-9e13-4679-aebe-247092f1d740 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:02.270837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611ff62-9e13-4679-aebe-247092f1d740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:02.271277 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611ff62-9e13-4679-aebe-247092f1d740 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:02.272775 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c611ff62-9e13-4679-aebe-247092f1d740 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:02.276471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c611ff62-9e13-4679-aebe-247092f1d740 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:02.317336 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1360/2752] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:10:02 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:10:02.322254 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd1460-6dd3-4851-97c8-9b97815e3d22 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:02.336254 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd1460-6dd3-4851-97c8-9b97815e3d22 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:02.341861 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a67d4-ae47-4e8b-b0a1-4df437cedf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:02.342463 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a67d4-ae47-4e8b-b0a1-4df437cedf1f admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:02.342810 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a67d4-ae47-4e8b-b0a1-4df437cedf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:02.343292 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a67d4-ae47-4e8b-b0a1-4df437cedf1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:02.344134 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a67d4-ae47-4e8b-b0a1-4df437cedf1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:02.346552 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a67d4-ae47-4e8b-b0a1-4df437cedf1f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:02.358323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1393/2753] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:10:02.363417 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c611ff62-9e13-4679-aebe-247092f1d740 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:02.375110 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c611ff62-9e13-4679-aebe-247092f1d740 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:02.381022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7abfa6-ce28-47c5-895d-41f8c5a7c658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:02.382008 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7abfa6-ce28-47c5-895d-41f8c5a7c658 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:02.382458 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7abfa6-ce28-47c5-895d-41f8c5a7c658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:02.382923 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7abfa6-ce28-47c5-895d-41f8c5a7c658 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:02.385730 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7abfa6-ce28-47c5-895d-41f8c5a7c658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:02.385931 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7abfa6-ce28-47c5-895d-41f8c5a7c658 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:02.402500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1361/2754] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:10:02.408499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a67d4-ae47-4e8b-b0a1-4df437cedf1f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:02.418413 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a67d4-ae47-4e8b-b0a1-4df437cedf1f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:02.423625 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42370a99-fcdd-47d1-92c6-7fb411d7e32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:02.424324 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42370a99-fcdd-47d1-92c6-7fb411d7e32f admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:02.424569 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42370a99-fcdd-47d1-92c6-7fb411d7e32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:02.425033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42370a99-fcdd-47d1-92c6-7fb411d7e32f admin admin] PATH_INFO: `/v3/projects/156f361e46de4268aa3ee1de41b34700/users/fea36f99c71649d49ece59174bbe8ae6/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:02.443184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:02.449012 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42370a99-fcdd-47d1-92c6-7fb411d7e32f admin admin] RBAC: Authorizing `identity:create_grant(project_id=156f361e46de4268aa3ee1de41b34700, user_id=fea36f99c71649d49ece59174bbe8ae6, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:02.452029 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42370a99-fcdd-47d1-92c6-7fb411d7e32f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:02.484601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1394/2755] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:10:02 2022] PUT /identity/v3/projects/156f361e46de4268aa3ee1de41b34700/users/fea36f99c71649d49ece59174bbe8ae6/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:10:02.486606 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f835c6-b121-4e48-b2ba-cc6072039b78 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:02.487791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f835c6-b121-4e48-b2ba-cc6072039b78 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:02.488766 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f835c6-b121-4e48-b2ba-cc6072039b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:02.524686 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-92f835c6-b121-4e48-b2ba-cc6072039b78 None None] MFA Rules not processed for user `fea36f99c71649d49ece59174bbe8ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:02.526147 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-92f835c6-b121-4e48-b2ba-cc6072039b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:02.593225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:02.620144 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1362/2756] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:10:02.634765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42370a99-fcdd-47d1-92c6-7fb411d7e32f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:02.636698 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-42370a99-fcdd-47d1-92c6-7fb411d7e32f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:02.708001 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42370a99-fcdd-47d1-92c6-7fb411d7e32f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:02.712003 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61187278-5ece-48c1-ac91-105536c25832 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:02.712648 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61187278-5ece-48c1-ac91-105536c25832 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:02.712932 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61187278-5ece-48c1-ac91-105536c25832 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:02.713309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61187278-5ece-48c1-ac91-105536c25832 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:02.732920 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61187278-5ece-48c1-ac91-105536c25832 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:02.734545 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61187278-5ece-48c1-ac91-105536c25832 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:02.747229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1395/2757] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:10:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:10:12.616615 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=6 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:12.741532 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=29,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:15.516664 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f835c6-b121-4e48-b2ba-cc6072039b78 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:15.518186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-92f835c6-b121-4e48-b2ba-cc6072039b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:15.633362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f835c6-b121-4e48-b2ba-cc6072039b78 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:15.639019 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04467e03-a581-459e-a167-a0bbc2d20f05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:15.639706 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04467e03-a581-459e-a167-a0bbc2d20f05 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:15.639995 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04467e03-a581-459e-a167-a0bbc2d20f05 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:15.640271 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04467e03-a581-459e-a167-a0bbc2d20f05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:15.642086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-04467e03-a581-459e-a167-a0bbc2d20f05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:15.654927 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04467e03-a581-459e-a167-a0bbc2d20f05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:15.656517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04467e03-a581-459e-a167-a0bbc2d20f05 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:15.667916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1363/2758] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:10:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:10:22.618884 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61187278-5ece-48c1-ac91-105536c25832 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:22.702765 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61187278-5ece-48c1-ac91-105536c25832 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:22.710465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba00df-f403-4d22-bed8-12df4ea74f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:22.712465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba00df-f403-4d22-bed8-12df4ea74f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:22.712465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba00df-f403-4d22-bed8-12df4ea74f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:22.712465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba00df-f403-4d22-bed8-12df4ea74f01 admin admin] PATH_INFO: `/v3/users/fea36f99c71649d49ece59174bbe8ae6` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:22.713351 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba00df-f403-4d22-bed8-12df4ea74f01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fea36f99c71649d49ece59174bbe8ae6)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:22.715448 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba00df-f403-4d22-bed8-12df4ea74f01 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:22.773616 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e8ba00df-f403-4d22-bed8-12df4ea74f01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fea36f99c71649d49ece59174bbe8ae6'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:22.774313 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e8ba00df-f403-4d22-bed8-12df4ea74f01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fea36f99c71649d49ece59174bbe8ae6'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:22.782731 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e8ba00df-f403-4d22-bed8-12df4ea74f01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fea36f99c71649d49ece59174bbe8ae6'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:22.797283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e8ba00df-f403-4d22-bed8-12df4ea74f01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fea36f99c71649d49ece59174bbe8ae6'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:22.804876 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e8ba00df-f403-4d22-bed8-12df4ea74f01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fea36f99c71649d49ece59174bbe8ae6'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:22.812086 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1396/2759] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:10:22 2022] DELETE /identity/v3/users/fea36f99c71649d49ece59174bbe8ae6 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:10:22.818315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752d322-f84c-4c33-a819-144a05f9290b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:22.818774 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752d322-f84c-4c33-a819-144a05f9290b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:22.818868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752d322-f84c-4c33-a819-144a05f9290b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:22.839587 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7752d322-f84c-4c33-a819-144a05f9290b None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:22.840951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7752d322-f84c-4c33-a819-144a05f9290b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:22.906906 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1364/2760] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:10:22.920342 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba00df-f403-4d22-bed8-12df4ea74f01 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:22.922199 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e8ba00df-f403-4d22-bed8-12df4ea74f01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:22.957580 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752d322-f84c-4c33-a819-144a05f9290b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:22.959049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7752d322-f84c-4c33-a819-144a05f9290b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:22.987351 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba00df-f403-4d22-bed8-12df4ea74f01 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:22.991837 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f0ae3-45ef-4963-9d39-84ed6a12e393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:22.992477 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f0ae3-45ef-4963-9d39-84ed6a12e393 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:22.992995 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f0ae3-45ef-4963-9d39-84ed6a12e393 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:22.993272 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f0ae3-45ef-4963-9d39-84ed6a12e393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:23.004796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f0ae3-45ef-4963-9d39-84ed6a12e393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:23.007480 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f0ae3-45ef-4963-9d39-84ed6a12e393 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:23.020033 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1397/2761] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:10:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:10:23.026843 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752d322-f84c-4c33-a819-144a05f9290b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:23.032585 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24471d-d32c-4cb4-bbfe-424a25be789b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:23.033232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24471d-d32c-4cb4-bbfe-424a25be789b service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:23.033517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24471d-d32c-4cb4-bbfe-424a25be789b service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:23.033807 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24471d-d32c-4cb4-bbfe-424a25be789b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:23.036122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1d24471d-d32c-4cb4-bbfe-424a25be789b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:23.090602 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:23.125251 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24471d-d32c-4cb4-bbfe-424a25be789b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:23.126768 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24471d-d32c-4cb4-bbfe-424a25be789b service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:23.137951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1365/2762] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:10:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:10:23.317667 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f0ae3-45ef-4963-9d39-84ed6a12e393 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:23.319300 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-848f0ae3-45ef-4963-9d39-84ed6a12e393 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:23.332824 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f0ae3-45ef-4963-9d39-84ed6a12e393 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:23.340554 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af2851-e5a4-4001-8538-8da380734070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:23.341231 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af2851-e5a4-4001-8538-8da380734070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:23.341521 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af2851-e5a4-4001-8538-8da380734070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:23.341797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af2851-e5a4-4001-8538-8da380734070 admin admin] PATH_INFO: `/v3/projects/156f361e46de4268aa3ee1de41b34700` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:23.343085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af2851-e5a4-4001-8538-8da380734070 admin admin] RBAC: Authorizing `identity:delete_project(project_id=156f361e46de4268aa3ee1de41b34700)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:23.344949 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af2851-e5a4-4001-8538-8da380734070 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:23.364122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:23.407344 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-79af2851-e5a4-4001-8538-8da380734070 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '156f361e46de4268aa3ee1de41b34700', 'request_id': 'req-79af2851-e5a4-4001-8538-8da380734070'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:23.413317 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-79af2851-e5a4-4001-8538-8da380734070 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '156f361e46de4268aa3ee1de41b34700', 'request_id': 'req-79af2851-e5a4-4001-8538-8da380734070'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:23.421757 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-79af2851-e5a4-4001-8538-8da380734070 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '156f361e46de4268aa3ee1de41b34700', 'request_id': 'req-79af2851-e5a4-4001-8538-8da380734070'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:23.440307 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-79af2851-e5a4-4001-8538-8da380734070 admin admin] The token cache is being invalidate because project 156f361e46de4268aa3ee1de41b34700 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:10:23.440739 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-79af2851-e5a4-4001-8538-8da380734070 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:23.442221 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-79af2851-e5a4-4001-8538-8da380734070 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:23.443937 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1398/2763] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:10:23 2022] DELETE /identity/v3/projects/156f361e46de4268aa3ee1de41b34700 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:10:23.450486 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42347cd8-bfb4-4ef0-bd73-5dba05c94a4d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:23.450835 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42347cd8-bfb4-4ef0-bd73-5dba05c94a4d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:23.451114 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42347cd8-bfb4-4ef0-bd73-5dba05c94a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:23.472847 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-42347cd8-bfb4-4ef0-bd73-5dba05c94a4d None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:23.474154 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-42347cd8-bfb4-4ef0-bd73-5dba05c94a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:23.544840 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1366/2764] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:10:23.553494 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68093d2e-2e9c-492b-b5e4-056f011939ff None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:23.553774 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68093d2e-2e9c-492b-b5e4-056f011939ff None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:23.554100 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68093d2e-2e9c-492b-b5e4-056f011939ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:23.575746 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-68093d2e-2e9c-492b-b5e4-056f011939ff None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:23.578330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-68093d2e-2e9c-492b-b5e4-056f011939ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:23.589611 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1399/2765] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:10:23.595858 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42347cd8-bfb4-4ef0-bd73-5dba05c94a4d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:23.609777 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42347cd8-bfb4-4ef0-bd73-5dba05c94a4d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:23.615733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c08c3-e2fb-4b07-829f-9f88807cd8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:23.616484 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c08c3-e2fb-4b07-829f-9f88807cd8c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:23.616918 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c08c3-e2fb-4b07-829f-9f88807cd8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:23.617199 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c08c3-e2fb-4b07-829f-9f88807cd8c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:23.617935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c08c3-e2fb-4b07-829f-9f88807cd8c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:10:23.618249 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c08c3-e2fb-4b07-829f-9f88807cd8c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:23.619854 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c08c3-e2fb-4b07-829f-9f88807cd8c8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:23.637077 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1367/2766] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:10:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:10:23.641521 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68093d2e-2e9c-492b-b5e4-056f011939ff None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:23.651433 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68093d2e-2e9c-492b-b5e4-056f011939ff None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:23.656178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2dcb2-dd6c-43b5-bf47-e223832d6607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:23.656801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2dcb2-dd6c-43b5-bf47-e223832d6607 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:23.657084 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2dcb2-dd6c-43b5-bf47-e223832d6607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:23.657450 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2dcb2-dd6c-43b5-bf47-e223832d6607 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:23.658295 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2dcb2-dd6c-43b5-bf47-e223832d6607 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:23.660723 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2dcb2-dd6c-43b5-bf47-e223832d6607 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:23.692184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1400/2767] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:10:23 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:10:23.696504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c08c3-e2fb-4b07-829f-9f88807cd8c8 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:23.711748 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c08c3-e2fb-4b07-829f-9f88807cd8c8 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:23.770606 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8b8c5-cb83-448c-8508-59fd6b5ae2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:23.771231 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8b8c5-cb83-448c-8508-59fd6b5ae2b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:23.771515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8b8c5-cb83-448c-8508-59fd6b5ae2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:23.771792 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8b8c5-cb83-448c-8508-59fd6b5ae2b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:23.772515 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8b8c5-cb83-448c-8508-59fd6b5ae2b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:23.774528 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8b8c5-cb83-448c-8508-59fd6b5ae2b0 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:23.798011 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1368/2768] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:10:23 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:10:23.803004 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2dcb2-dd6c-43b5-bf47-e223832d6607 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:23.813195 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2dcb2-dd6c-43b5-bf47-e223832d6607 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:23.817866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33175c-3d53-4b65-bd13-4e2f4c580aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:23.818589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33175c-3d53-4b65-bd13-4e2f4c580aba admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:23.818866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33175c-3d53-4b65-bd13-4e2f4c580aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:23.819138 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33175c-3d53-4b65-bd13-4e2f4c580aba admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:23.819868 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33175c-3d53-4b65-bd13-4e2f4c580aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:23.821445 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33175c-3d53-4b65-bd13-4e2f4c580aba admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:23.832609 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1401/2769] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:10:23.838573 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8b8c5-cb83-448c-8508-59fd6b5ae2b0 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:23.851103 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8b8c5-cb83-448c-8508-59fd6b5ae2b0 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:23.856205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df3866-ce3b-44ad-a61e-0c72dbc908cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:23.856842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df3866-ce3b-44ad-a61e-0c72dbc908cd admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:23.857128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df3866-ce3b-44ad-a61e-0c72dbc908cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:23.857403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df3866-ce3b-44ad-a61e-0c72dbc908cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:23.858150 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df3866-ce3b-44ad-a61e-0c72dbc908cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:23.859775 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df3866-ce3b-44ad-a61e-0c72dbc908cd admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:23.877475 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1369/2770] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:10:23.879523 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33175c-3d53-4b65-bd13-4e2f4c580aba admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:23.890384 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33175c-3d53-4b65-bd13-4e2f4c580aba admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:23.898164 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f00026-7924-40e1-800f-12ab3600199b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:23.898978 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f00026-7924-40e1-800f-12ab3600199b admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:23.899282 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f00026-7924-40e1-800f-12ab3600199b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:23.899551 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f00026-7924-40e1-800f-12ab3600199b admin admin] PATH_INFO: `/v3/projects/a565fd14b3984b209d5684fa281139d7/users/9a840a71b1454fb89e1ec8aeb72f0542/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:23.916832 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f00026-7924-40e1-800f-12ab3600199b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a565fd14b3984b209d5684fa281139d7, user_id=9a840a71b1454fb89e1ec8aeb72f0542, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:23.919593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f00026-7924-40e1-800f-12ab3600199b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:23.933060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1402/2771] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:10:23 2022] PUT /identity/v3/projects/a565fd14b3984b209d5684fa281139d7/users/9a840a71b1454fb89e1ec8aeb72f0542/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:10:23.945234 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb64e2-4efd-4952-a0e5-722f0bfd8d5d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:23.945522 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb64e2-4efd-4952-a0e5-722f0bfd8d5d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:23.945831 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb64e2-4efd-4952-a0e5-722f0bfd8d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:23.977675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-86cb64e2-4efd-4952-a0e5-722f0bfd8d5d None None] MFA Rules not processed for user `9a840a71b1454fb89e1ec8aeb72f0542`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:23.979297 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-86cb64e2-4efd-4952-a0e5-722f0bfd8d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:24.006837 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8159e-8540-49ab-958f-29243b979ba7 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:24.007120 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8159e-8540-49ab-958f-29243b979ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:24.007391 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8159e-8540-49ab-958f-29243b979ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:24.010082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:24.057094 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1370/2772] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:10:24.068536 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb64e2-4efd-4952-a0e5-722f0bfd8d5d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:24.069953 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-86cb64e2-4efd-4952-a0e5-722f0bfd8d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:24.079283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-29b8159e-8540-49ab-958f-29243b979ba7 None None] MFA Rules not processed for user `f36c6987ef774a1684c55f885e2a4f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:24.080571 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-29b8159e-8540-49ab-958f-29243b979ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:24.137316 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb64e2-4efd-4952-a0e5-722f0bfd8d5d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:24.141186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0889ea0-4b08-4392-8652-d90003328bcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:24.141813 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0889ea0-4b08-4392-8652-d90003328bcf service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:24.142139 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0889ea0-4b08-4392-8652-d90003328bcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:24.142466 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0889ea0-4b08-4392-8652-d90003328bcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:24.152596 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0889ea0-4b08-4392-8652-d90003328bcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:24.154111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0889ea0-4b08-4392-8652-d90003328bcf service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:24.160974 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1403/2773] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Aug 1 16:10:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:10:24.164412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1371/2774] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:10:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:10:24.217770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8159e-8540-49ab-958f-29243b979ba7 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:24.219306 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-29b8159e-8540-49ab-958f-29243b979ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:24.224550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:24.337356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8159e-8540-49ab-958f-29243b979ba7 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:24.342703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361b6dd-5318-4b88-ab6b-8d0058e2f0da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6873a3e440bd4b63b0082fc85beef49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:24.343340 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361b6dd-5318-4b88-ab6b-8d0058e2f0da service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:24.343616 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361b6dd-5318-4b88-ab6b-8d0058e2f0da service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:24.343895 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361b6dd-5318-4b88-ab6b-8d0058e2f0da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:24.353028 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0361b6dd-5318-4b88-ab6b-8d0058e2f0da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:24.354764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0361b6dd-5318-4b88-ab6b-8d0058e2f0da service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:24.364419 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1404/2775] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Aug 1 16:10:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:10:24.958628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226369bd-6147-4745-8b0e-13464d07325b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:24.959109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226369bd-6147-4745-8b0e-13464d07325b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:24.959193 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226369bd-6147-4745-8b0e-13464d07325b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:24.980286 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-226369bd-6147-4745-8b0e-13464d07325b None None] MFA Rules not processed for user `f36c6987ef774a1684c55f885e2a4f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:24.981510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-226369bd-6147-4745-8b0e-13464d07325b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:24.991756 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1372/2776] 10.222.0.83 () {58 vars in 967 bytes} [Mon Aug 1 16:10:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:10:25.005196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361b6dd-5318-4b88-ab6b-8d0058e2f0da service swift] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:25.016434 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361b6dd-5318-4b88-ab6b-8d0058e2f0da service swift] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:25.022134 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a44b1d-8069-4ca2-9436-9f8f4ebbe593 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6873a3e440bd4b63b0082fc85beef49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:25.022670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a44b1d-8069-4ca2-9436-9f8f4ebbe593 service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:25.022960 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a44b1d-8069-4ca2-9436-9f8f4ebbe593 service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:25.023292 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a44b1d-8069-4ca2-9436-9f8f4ebbe593 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:25.033129 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a44b1d-8069-4ca2-9436-9f8f4ebbe593 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:25.035290 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a44b1d-8069-4ca2-9436-9f8f4ebbe593 service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:25.044323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1405/2777] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Aug 1 16:10:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:10:25.277412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36151488-294d-46da-b04d-19e704919109 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:25.277700 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36151488-294d-46da-b04d-19e704919109 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:25.278008 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36151488-294d-46da-b04d-19e704919109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:25.300923 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-36151488-294d-46da-b04d-19e704919109 None None] MFA Rules not processed for user `f36c6987ef774a1684c55f885e2a4f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:25.302286 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-36151488-294d-46da-b04d-19e704919109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:25.310481 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1373/2778] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Aug 1 16:10:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:10:25.323258 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a44b1d-8069-4ca2-9436-9f8f4ebbe593 service swift] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:25.335198 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a44b1d-8069-4ca2-9436-9f8f4ebbe593 service swift] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:25.340453 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4911e48-7d48-4021-a9de-fdbe9db4f76b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6873a3e440bd4b63b0082fc85beef49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:25.341163 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4911e48-7d48-4021-a9de-fdbe9db4f76b service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:25.341450 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4911e48-7d48-4021-a9de-fdbe9db4f76b service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:25.341718 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4911e48-7d48-4021-a9de-fdbe9db4f76b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:25.352055 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4911e48-7d48-4021-a9de-fdbe9db4f76b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:25.353643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4911e48-7d48-4021-a9de-fdbe9db4f76b service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:25.362106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1406/2779] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Aug 1 16:10:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:10:25.706231 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cadd81-def1-4f69-b87b-8d6af3a392d6 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:25.706630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cadd81-def1-4f69-b87b-8d6af3a392d6 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:25.706968 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cadd81-def1-4f69-b87b-8d6af3a392d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:25.726738 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d7cadd81-def1-4f69-b87b-8d6af3a392d6 None None] MFA Rules not processed for user `f36c6987ef774a1684c55f885e2a4f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:25.728097 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d7cadd81-def1-4f69-b87b-8d6af3a392d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:25.735962 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1374/2780] 10.222.0.83 () {58 vars in 967 bytes} [Mon Aug 1 16:10:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:10:25.754307 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4911e48-7d48-4021-a9de-fdbe9db4f76b service swift] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:25.765492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4911e48-7d48-4021-a9de-fdbe9db4f76b service swift] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:25.770863 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7aa70-5fb6-4bd9-812d-e1bb21bd82ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6873a3e440bd4b63b0082fc85beef49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:25.771493 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7aa70-5fb6-4bd9-812d-e1bb21bd82ef service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:25.771771 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7aa70-5fb6-4bd9-812d-e1bb21bd82ef service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:25.772043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7aa70-5fb6-4bd9-812d-e1bb21bd82ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:25.781397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7aa70-5fb6-4bd9-812d-e1bb21bd82ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:25.783048 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7aa70-5fb6-4bd9-812d-e1bb21bd82ef service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:25.793476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1407/2781] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Aug 1 16:10:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:10:27.588178 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cadd81-def1-4f69-b87b-8d6af3a392d6 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:27.664906 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cadd81-def1-4f69-b87b-8d6af3a392d6 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:27.669376 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6934d0-412f-44bc-927f-95c988c48619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:27.670052 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6934d0-412f-44bc-927f-95c988c48619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:27.670339 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6934d0-412f-44bc-927f-95c988c48619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:27.670612 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6934d0-412f-44bc-927f-95c988c48619 admin admin] PATH_INFO: `/v3/users/9a840a71b1454fb89e1ec8aeb72f0542` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:27.671803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6934d0-412f-44bc-927f-95c988c48619 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a840a71b1454fb89e1ec8aeb72f0542)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:27.673716 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6934d0-412f-44bc-927f-95c988c48619 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:27.722242 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ef6934d0-412f-44bc-927f-95c988c48619 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a840a71b1454fb89e1ec8aeb72f0542'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:27.722998 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ef6934d0-412f-44bc-927f-95c988c48619 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a840a71b1454fb89e1ec8aeb72f0542'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:27.725787 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:27.731716 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ef6934d0-412f-44bc-927f-95c988c48619 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a840a71b1454fb89e1ec8aeb72f0542'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:27.746994 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ef6934d0-412f-44bc-927f-95c988c48619 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a840a71b1454fb89e1ec8aeb72f0542'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:27.747759 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ef6934d0-412f-44bc-927f-95c988c48619 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a840a71b1454fb89e1ec8aeb72f0542'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:27.760997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1375/2782] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:10:27 2022] DELETE /identity/v3/users/9a840a71b1454fb89e1ec8aeb72f0542 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:10:27.765835 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796da84-823e-42de-b6e3-b9de5cd6d439 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:27.766318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796da84-823e-42de-b6e3-b9de5cd6d439 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:27.766547 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796da84-823e-42de-b6e3-b9de5cd6d439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:27.785813 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1796da84-823e-42de-b6e3-b9de5cd6d439 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:27.787072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1796da84-823e-42de-b6e3-b9de5cd6d439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:27.849502 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1408/2783] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:10:27.859860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6934d0-412f-44bc-927f-95c988c48619 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:27.861549 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ef6934d0-412f-44bc-927f-95c988c48619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:27.934412 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6934d0-412f-44bc-927f-95c988c48619 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:27.938220 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec9bc4-0494-4915-b831-69f1ce435de3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:27.938859 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec9bc4-0494-4915-b831-69f1ce435de3 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:27.939146 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec9bc4-0494-4915-b831-69f1ce435de3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:27.939536 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec9bc4-0494-4915-b831-69f1ce435de3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:27.950483 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec9bc4-0494-4915-b831-69f1ce435de3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:27.951940 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec9bc4-0494-4915-b831-69f1ce435de3 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:27.962208 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1376/2784] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:10:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:10:28.116553 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796da84-823e-42de-b6e3-b9de5cd6d439 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:28.118075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1796da84-823e-42de-b6e3-b9de5cd6d439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:28.131307 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796da84-823e-42de-b6e3-b9de5cd6d439 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:28.136455 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb6e07-e03a-42a0-ba62-907919482e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:28.137131 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb6e07-e03a-42a0-ba62-907919482e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:28.137436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb6e07-e03a-42a0-ba62-907919482e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:28.137674 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb6e07-e03a-42a0-ba62-907919482e5d admin admin] PATH_INFO: `/v3/users/1026208ad63143f589f05818ce082174` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:28.138897 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb6e07-e03a-42a0-ba62-907919482e5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1026208ad63143f589f05818ce082174)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:28.140815 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb6e07-e03a-42a0-ba62-907919482e5d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:28.183255 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:28.190340 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec9bc4-0494-4915-b831-69f1ce435de3 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:28.193463 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-88ec9bc4-0494-4915-b831-69f1ce435de3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:28.211986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7ccb6e07-e03a-42a0-ba62-907919482e5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1026208ad63143f589f05818ce082174'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:28.212971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7ccb6e07-e03a-42a0-ba62-907919482e5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1026208ad63143f589f05818ce082174'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:28.213308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec9bc4-0494-4915-b831-69f1ce435de3 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:28.220087 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d86a3b-9563-4b05-b0d9-85fe861fa88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:28.221012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d86a3b-9563-4b05-b0d9-85fe861fa88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:28.221435 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d86a3b-9563-4b05-b0d9-85fe861fa88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:28.221639 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d86a3b-9563-4b05-b0d9-85fe861fa88f admin admin] PATH_INFO: `/v3/users/1510e52b41f545e4b61d310a08c23a3e` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:28.223575 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d86a3b-9563-4b05-b0d9-85fe861fa88f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1510e52b41f545e4b61d310a08c23a3e)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:28.225650 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d86a3b-9563-4b05-b0d9-85fe861fa88f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:28.228320 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7ccb6e07-e03a-42a0-ba62-907919482e5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1026208ad63143f589f05818ce082174'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:28.260904 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7ccb6e07-e03a-42a0-ba62-907919482e5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1026208ad63143f589f05818ce082174'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:28.271458 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7ccb6e07-e03a-42a0-ba62-907919482e5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1026208ad63143f589f05818ce082174'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:28.280223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1409/2785] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:10:28 2022] DELETE /identity/v3/users/1026208ad63143f589f05818ce082174 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:10:28.285739 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2d452-6489-4ce8-ab7e-0e2cabb019bc None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:28.286075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2d452-6489-4ce8-ab7e-0e2cabb019bc None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:28.286340 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2d452-6489-4ce8-ab7e-0e2cabb019bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:28.310354 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e9d86a3b-9563-4b05-b0d9-85fe861fa88f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1510e52b41f545e4b61d310a08c23a3e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:28.310492 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-b3c2d452-6489-4ce8-ab7e-0e2cabb019bc None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:28.311480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e9d86a3b-9563-4b05-b0d9-85fe861fa88f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1510e52b41f545e4b61d310a08c23a3e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:28.312164 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b3c2d452-6489-4ce8-ab7e-0e2cabb019bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:28.325150 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e9d86a3b-9563-4b05-b0d9-85fe861fa88f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1510e52b41f545e4b61d310a08c23a3e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:28.342122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e9d86a3b-9563-4b05-b0d9-85fe861fa88f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1510e52b41f545e4b61d310a08c23a3e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:28.343018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e9d86a3b-9563-4b05-b0d9-85fe861fa88f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1510e52b41f545e4b61d310a08c23a3e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:28.361509 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1377/2786] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:10:28 2022] DELETE /identity/v3/users/1510e52b41f545e4b61d310a08c23a3e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:10:28.363148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d86a3b-9563-4b05-b0d9-85fe861fa88f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:28.364646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e9d86a3b-9563-4b05-b0d9-85fe861fa88f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:28.391567 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1410/2787] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:10:28.392965 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea54a9-d465-4e8a-a9e6-018233b5c22d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:28.393319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea54a9-d465-4e8a-a9e6-018233b5c22d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:28.394202 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea54a9-d465-4e8a-a9e6-018233b5c22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:28.414616 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-26ea54a9-d465-4e8a-a9e6-018233b5c22d None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:28.415829 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-26ea54a9-d465-4e8a-a9e6-018233b5c22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:28.428687 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:28.451888 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d86a3b-9563-4b05-b0d9-85fe861fa88f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:28.457933 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54db40-c57b-4b29-98ac-d21ec5d92d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:28.458599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54db40-c57b-4b29-98ac-d21ec5d92d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:28.458883 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54db40-c57b-4b29-98ac-d21ec5d92d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:28.459155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54db40-c57b-4b29-98ac-d21ec5d92d48 admin admin] PATH_INFO: `/v3/projects/a565fd14b3984b209d5684fa281139d7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:28.460409 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54db40-c57b-4b29-98ac-d21ec5d92d48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a565fd14b3984b209d5684fa281139d7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:28.462380 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54db40-c57b-4b29-98ac-d21ec5d92d48 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:28.492529 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1411/2788] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:10:28.492529 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea54a9-d465-4e8a-a9e6-018233b5c22d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:28.493139 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-26ea54a9-d465-4e8a-a9e6-018233b5c22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:28.524839 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9b54db40-c57b-4b29-98ac-d21ec5d92d48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a565fd14b3984b209d5684fa281139d7', 'request_id': 'req-9b54db40-c57b-4b29-98ac-d21ec5d92d48'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:28.531654 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9b54db40-c57b-4b29-98ac-d21ec5d92d48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a565fd14b3984b209d5684fa281139d7', 'request_id': 'req-9b54db40-c57b-4b29-98ac-d21ec5d92d48'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:28.538597 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9b54db40-c57b-4b29-98ac-d21ec5d92d48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a565fd14b3984b209d5684fa281139d7', 'request_id': 'req-9b54db40-c57b-4b29-98ac-d21ec5d92d48'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:28.566232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9b54db40-c57b-4b29-98ac-d21ec5d92d48 admin admin] The token cache is being invalidate because project a565fd14b3984b209d5684fa281139d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:10:28.566232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9b54db40-c57b-4b29-98ac-d21ec5d92d48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:28.566232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-9b54db40-c57b-4b29-98ac-d21ec5d92d48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:28.566232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1378/2789] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:10:28 2022] DELETE /identity/v3/projects/a565fd14b3984b209d5684fa281139d7 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:10:28.566232 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54db40-c57b-4b29-98ac-d21ec5d92d48 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:28.566802 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9b54db40-c57b-4b29-98ac-d21ec5d92d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:28.582951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:28.589859 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea54a9-d465-4e8a-a9e6-018233b5c22d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:28.663260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54db40-c57b-4b29-98ac-d21ec5d92d48 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:28.668013 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a62d9f-87ec-418a-a3bb-3408ac272ad1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:28.670601 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a62d9f-87ec-418a-a3bb-3408ac272ad1 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:28.670601 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a62d9f-87ec-418a-a3bb-3408ac272ad1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:28.670601 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a62d9f-87ec-418a-a3bb-3408ac272ad1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:28.673186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-17a62d9f-87ec-418a-a3bb-3408ac272ad1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:28.689400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947b821-78e1-459d-931a-a65807bdcb9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:28.689400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947b821-78e1-459d-931a-a65807bdcb9d service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:28.689400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947b821-78e1-459d-931a-a65807bdcb9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:28.689400 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947b821-78e1-459d-931a-a65807bdcb9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:28.692064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4947b821-78e1-459d-931a-a65807bdcb9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:28.764858 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a62d9f-87ec-418a-a3bb-3408ac272ad1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:28.766324 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a62d9f-87ec-418a-a3bb-3408ac272ad1 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:28.794027 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1379/2790] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:10:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:10:28.808896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947b821-78e1-459d-931a-a65807bdcb9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:28.810590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947b821-78e1-459d-931a-a65807bdcb9d service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:28.822772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1412/2791] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:10:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:10:29.063011 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a62d9f-87ec-418a-a3bb-3408ac272ad1 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:29.064600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-17a62d9f-87ec-418a-a3bb-3408ac272ad1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:29.098980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a62d9f-87ec-418a-a3bb-3408ac272ad1 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:29.109033 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f33937-bb3e-44bf-905a-63b59d1b3ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:29.109704 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f33937-bb3e-44bf-905a-63b59d1b3ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:29.110024 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f33937-bb3e-44bf-905a-63b59d1b3ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:29.110306 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f33937-bb3e-44bf-905a-63b59d1b3ba1 admin admin] PATH_INFO: `/v3/projects/53f9adda77ff445b8eecd09fa7184530` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:29.111522 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f33937-bb3e-44bf-905a-63b59d1b3ba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53f9adda77ff445b8eecd09fa7184530)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:29.114724 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f33937-bb3e-44bf-905a-63b59d1b3ba1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:29.134853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:29.149108 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947b821-78e1-459d-931a-a65807bdcb9d service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:29.151394 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4947b821-78e1-459d-931a-a65807bdcb9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:29.174605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947b821-78e1-459d-931a-a65807bdcb9d service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:29.178195 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c5f33937-bb3e-44bf-905a-63b59d1b3ba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53f9adda77ff445b8eecd09fa7184530', 'request_id': 'req-c5f33937-bb3e-44bf-905a-63b59d1b3ba1'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:29.188264 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c5f33937-bb3e-44bf-905a-63b59d1b3ba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53f9adda77ff445b8eecd09fa7184530', 'request_id': 'req-c5f33937-bb3e-44bf-905a-63b59d1b3ba1'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:29.198033 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c5f33937-bb3e-44bf-905a-63b59d1b3ba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53f9adda77ff445b8eecd09fa7184530', 'request_id': 'req-c5f33937-bb3e-44bf-905a-63b59d1b3ba1'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:29.214719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c5f33937-bb3e-44bf-905a-63b59d1b3ba1 admin admin] The token cache is being invalidate because project 53f9adda77ff445b8eecd09fa7184530 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:10:29.215189 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c5f33937-bb3e-44bf-905a-63b59d1b3ba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:29.216169 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c5f33937-bb3e-44bf-905a-63b59d1b3ba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:29.218267 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1380/2792] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:10:29 2022] DELETE /identity/v3/projects/53f9adda77ff445b8eecd09fa7184530 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:10:29.224244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4de20-744c-447c-850e-c3a8990ee931 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:29.224531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4de20-744c-447c-850e-c3a8990ee931 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:29.225048 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4de20-744c-447c-850e-c3a8990ee931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:29.248711 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-51e4de20-744c-447c-850e-c3a8990ee931 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:29.250160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-51e4de20-744c-447c-850e-c3a8990ee931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:29.253902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907628e-e533-4910-ad5d-0d6ec3362c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:29.254588 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907628e-e533-4910-ad5d-0d6ec3362c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:29.254873 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907628e-e533-4910-ad5d-0d6ec3362c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:29.255147 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907628e-e533-4910-ad5d-0d6ec3362c91 admin admin] PATH_INFO: `/v3/projects/030a3645d6d24ed4b10bd4b423b0f24b` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:29.256320 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907628e-e533-4910-ad5d-0d6ec3362c91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=030a3645d6d24ed4b10bd4b423b0f24b)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:29.258344 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907628e-e533-4910-ad5d-0d6ec3362c91 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:29.259173 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1381/2793] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:10:29.267651 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e19d3-525d-4cee-83b0-44884072b4e7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:29.267836 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e19d3-525d-4cee-83b0-44884072b4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:29.268021 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e19d3-525d-4cee-83b0-44884072b4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:29.291085 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-972e19d3-525d-4cee-83b0-44884072b4e7 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:29.293990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-972e19d3-525d-4cee-83b0-44884072b4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:29.303233 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1382/2794] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:10:29.307457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e19d3-525d-4cee-83b0-44884072b4e7 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:29.313880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1907628e-e533-4910-ad5d-0d6ec3362c91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '030a3645d6d24ed4b10bd4b423b0f24b', 'request_id': 'req-1907628e-e533-4910-ad5d-0d6ec3362c91'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:29.316865 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e19d3-525d-4cee-83b0-44884072b4e7 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:29.319186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:29.321590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1907628e-e533-4910-ad5d-0d6ec3362c91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '030a3645d6d24ed4b10bd4b423b0f24b', 'request_id': 'req-1907628e-e533-4910-ad5d-0d6ec3362c91'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:29.332000 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1907628e-e533-4910-ad5d-0d6ec3362c91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '030a3645d6d24ed4b10bd4b423b0f24b', 'request_id': 'req-1907628e-e533-4910-ad5d-0d6ec3362c91'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:29.354919 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1907628e-e533-4910-ad5d-0d6ec3362c91 admin admin] The token cache is being invalidate because project 030a3645d6d24ed4b10bd4b423b0f24b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:10:29.355365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1907628e-e533-4910-ad5d-0d6ec3362c91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:29.356249 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1907628e-e533-4910-ad5d-0d6ec3362c91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:10:29.357842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1413/2795] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:10:29 2022] DELETE /identity/v3/projects/030a3645d6d24ed4b10bd4b423b0f24b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:10:29.398859 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f2e24-0ab2-4080-8509-870d7f90fdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:29.399824 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f2e24-0ab2-4080-8509-870d7f90fdb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:29.400357 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f2e24-0ab2-4080-8509-870d7f90fdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:29.400745 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f2e24-0ab2-4080-8509-870d7f90fdb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:29.401959 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f2e24-0ab2-4080-8509-870d7f90fdb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:10:29.402573 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f2e24-0ab2-4080-8509-870d7f90fdb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:29.404419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f2e24-0ab2-4080-8509-870d7f90fdb7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:29.423666 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1383/2796] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:10:29.428550 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907628e-e533-4910-ad5d-0d6ec3362c91 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:29.430437 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1907628e-e533-4910-ad5d-0d6ec3362c91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:29.444674 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907628e-e533-4910-ad5d-0d6ec3362c91 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:29.449622 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476362c1-1fd3-40c8-a8a3-c5a83584423f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:29.450393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476362c1-1fd3-40c8-a8a3-c5a83584423f admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:29.450670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476362c1-1fd3-40c8-a8a3-c5a83584423f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:29.450940 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476362c1-1fd3-40c8-a8a3-c5a83584423f admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:29.451628 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476362c1-1fd3-40c8-a8a3-c5a83584423f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:29.453582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476362c1-1fd3-40c8-a8a3-c5a83584423f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:29.495088 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1414/2797] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:10:29 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:10:29.499526 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f2e24-0ab2-4080-8509-870d7f90fdb7 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:29.511263 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f2e24-0ab2-4080-8509-870d7f90fdb7 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:29.580284 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3d7cd-f676-4605-8928-376675c426aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:29.581053 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3d7cd-f676-4605-8928-376675c426aa admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:29.582118 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3d7cd-f676-4605-8928-376675c426aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:29.582522 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3d7cd-f676-4605-8928-376675c426aa admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:29.583403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff3d7cd-f676-4605-8928-376675c426aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:29.585670 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff3d7cd-f676-4605-8928-376675c426aa admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:29.616007 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1384/2798] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:10:29 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:10:29.616475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476362c1-1fd3-40c8-a8a3-c5a83584423f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:29.630193 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476362c1-1fd3-40c8-a8a3-c5a83584423f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:29.635247 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3199c-2c52-4ffe-a7d6-28382f021064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:29.636014 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3199c-2c52-4ffe-a7d6-28382f021064 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:29.636302 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3199c-2c52-4ffe-a7d6-28382f021064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:29.636575 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3199c-2c52-4ffe-a7d6-28382f021064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:29.637801 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3199c-2c52-4ffe-a7d6-28382f021064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:29.640257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3199c-2c52-4ffe-a7d6-28382f021064 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:29.651888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1415/2799] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:10:29.656093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3d7cd-f676-4605-8928-376675c426aa admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:29.669537 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3d7cd-f676-4605-8928-376675c426aa admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:29.674517 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994b2c8-07fe-46bf-9fdc-5d20aef0b829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:29.675186 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994b2c8-07fe-46bf-9fdc-5d20aef0b829 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:29.675471 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994b2c8-07fe-46bf-9fdc-5d20aef0b829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:29.675751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994b2c8-07fe-46bf-9fdc-5d20aef0b829 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:29.676457 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e994b2c8-07fe-46bf-9fdc-5d20aef0b829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:29.678195 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e994b2c8-07fe-46bf-9fdc-5d20aef0b829 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:29.689443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1385/2800] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:10:29.695420 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3199c-2c52-4ffe-a7d6-28382f021064 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:29.704965 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3199c-2c52-4ffe-a7d6-28382f021064 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:29.710312 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16badf-62fc-4f30-ad5a-e3de03bac277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:29.710980 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16badf-62fc-4f30-ad5a-e3de03bac277 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:29.711257 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16badf-62fc-4f30-ad5a-e3de03bac277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:29.711528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16badf-62fc-4f30-ad5a-e3de03bac277 admin admin] PATH_INFO: `/v3/projects/c12ec1d14cb44902bed525172feed2f8/users/c1e71f7596824b7e9c7a88a22a6664c3/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:29.727435 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16badf-62fc-4f30-ad5a-e3de03bac277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c12ec1d14cb44902bed525172feed2f8, user_id=c1e71f7596824b7e9c7a88a22a6664c3, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:29.730134 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16badf-62fc-4f30-ad5a-e3de03bac277 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:29.745584 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1416/2801] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:10:29 2022] PUT /identity/v3/projects/c12ec1d14cb44902bed525172feed2f8/users/c1e71f7596824b7e9c7a88a22a6664c3/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:10:29.750675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a0de9-105a-43b9-b485-6db49d205c9c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:29.750967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a0de9-105a-43b9-b485-6db49d205c9c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:29.751249 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a0de9-105a-43b9-b485-6db49d205c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:29.760236 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:29.773649 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7d6a0de9-105a-43b9-b485-6db49d205c9c None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:29.775088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7d6a0de9-105a-43b9-b485-6db49d205c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:29.835920 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1386/2802] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:10:29.846411 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16badf-62fc-4f30-ad5a-e3de03bac277 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:29.848225 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9b16badf-62fc-4f30-ad5a-e3de03bac277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:29.909750 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16badf-62fc-4f30-ad5a-e3de03bac277 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:29.911539 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480085f1-0723-4368-af30-af97737ec4e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:29.912172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480085f1-0723-4368-af30-af97737ec4e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:29.912457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480085f1-0723-4368-af30-af97737ec4e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:29.912729 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480085f1-0723-4368-af30-af97737ec4e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:29.923237 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480085f1-0723-4368-af30-af97737ec4e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:29.924692 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480085f1-0723-4368-af30-af97737ec4e1 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:29.935491 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1417/2803] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:10:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:10:34.244382 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09524795-022a-4826-b1b7-b79970e75e3a None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:34.244876 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09524795-022a-4826-b1b7-b79970e75e3a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:34.244964 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09524795-022a-4826-b1b7-b79970e75e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:34.279381 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-09524795-022a-4826-b1b7-b79970e75e3a None None] MFA Rules not processed for user `c1e71f7596824b7e9c7a88a22a6664c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:34.280627 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-09524795-022a-4826-b1b7-b79970e75e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:34.354211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1387/2804] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:10:34.358896 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480085f1-0723-4368-af30-af97737ec4e1 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:34.369866 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480085f1-0723-4368-af30-af97737ec4e1 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:34.374723 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08acc4-b9a2-46fb-b3f3-8cde9ceadbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:34.375356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08acc4-b9a2-46fb-b3f3-8cde9ceadbb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:34.375634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08acc4-b9a2-46fb-b3f3-8cde9ceadbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:34.375907 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08acc4-b9a2-46fb-b3f3-8cde9ceadbb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:34.376585 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08acc4-b9a2-46fb-b3f3-8cde9ceadbb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:34.379459 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08acc4-b9a2-46fb-b3f3-8cde9ceadbb1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:34.410119 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1418/2805] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:10:34 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:10:34.416704 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09524795-022a-4826-b1b7-b79970e75e3a None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:34.431586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09524795-022a-4826-b1b7-b79970e75e3a None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:34.497940 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8a202-0f14-4ea5-a39e-c347b01f47fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:34.498604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8a202-0f14-4ea5-a39e-c347b01f47fc admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:34.498887 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8a202-0f14-4ea5-a39e-c347b01f47fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:34.499187 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8a202-0f14-4ea5-a39e-c347b01f47fc admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:34.499960 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8a202-0f14-4ea5-a39e-c347b01f47fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:34.501774 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8a202-0f14-4ea5-a39e-c347b01f47fc admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:34.526122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1388/2806] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:10:34 2022] POST /identity/v3/users => generated 295 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:10:34.530762 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08acc4-b9a2-46fb-b3f3-8cde9ceadbb1 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:34.541874 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08acc4-b9a2-46fb-b3f3-8cde9ceadbb1 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:34.546717 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43164e33-6591-4220-9073-cbbf319e9ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:34.547352 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43164e33-6591-4220-9073-cbbf319e9ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:34.547653 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43164e33-6591-4220-9073-cbbf319e9ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:34.547912 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43164e33-6591-4220-9073-cbbf319e9ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:34.548648 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43164e33-6591-4220-9073-cbbf319e9ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:34.550936 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43164e33-6591-4220-9073-cbbf319e9ca5 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:34.562202 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1419/2807] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:10:34.567109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8a202-0f14-4ea5-a39e-c347b01f47fc admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:34.577548 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8a202-0f14-4ea5-a39e-c347b01f47fc admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:34.582470 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf95a40-e77d-4481-9186-1d71dc6fa348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:34.583148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf95a40-e77d-4481-9186-1d71dc6fa348 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:34.583534 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf95a40-e77d-4481-9186-1d71dc6fa348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:34.583719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf95a40-e77d-4481-9186-1d71dc6fa348 admin admin] PATH_INFO: `/v3/projects/3e3600e9a58647e398779fba36146127/users/b695d7a8c77b4250921140b894acae0d/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:34.599433 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf95a40-e77d-4481-9186-1d71dc6fa348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e3600e9a58647e398779fba36146127, user_id=b695d7a8c77b4250921140b894acae0d, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:34.602136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf95a40-e77d-4481-9186-1d71dc6fa348 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:34.611936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:34.617961 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1389/2808] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:10:34 2022] PUT /identity/v3/projects/3e3600e9a58647e398779fba36146127/users/b695d7a8c77b4250921140b894acae0d/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:10:34.622065 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43164e33-6591-4220-9073-cbbf319e9ca5 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:34.633043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43164e33-6591-4220-9073-cbbf319e9ca5 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:34.685263 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:34.692786 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd0fc9-510b-4b0a-a852-1399cb7ab416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:34.693603 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd0fc9-510b-4b0a-a852-1399cb7ab416 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:34.694104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd0fc9-510b-4b0a-a852-1399cb7ab416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:34.694382 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd0fc9-510b-4b0a-a852-1399cb7ab416 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:34.695143 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd0fc9-510b-4b0a-a852-1399cb7ab416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:34.697231 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd0fc9-510b-4b0a-a852-1399cb7ab416 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:34.708756 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1420/2809] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:10:34.713082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf95a40-e77d-4481-9186-1d71dc6fa348 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:34.728413 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf95a40-e77d-4481-9186-1d71dc6fa348 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:34.734184 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9520a8-2ef9-419a-85d5-98b4782b69a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:34.734904 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9520a8-2ef9-419a-85d5-98b4782b69a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:34.735216 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9520a8-2ef9-419a-85d5-98b4782b69a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:34.735499 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9520a8-2ef9-419a-85d5-98b4782b69a3 admin admin] PATH_INFO: `/v3/projects/3e3600e9a58647e398779fba36146127/users/b695d7a8c77b4250921140b894acae0d/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:34.738205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9520a8-2ef9-419a-85d5-98b4782b69a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e3600e9a58647e398779fba36146127, user_id=b695d7a8c77b4250921140b894acae0d, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:34.741156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9520a8-2ef9-419a-85d5-98b4782b69a3 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:34.752375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1390/2810] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:10:34 2022] PUT /identity/v3/projects/3e3600e9a58647e398779fba36146127/users/b695d7a8c77b4250921140b894acae0d/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:10:39.937354 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6b6c8-80e1-4f2d-9159-e682dfdec09c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:39.937825 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6b6c8-80e1-4f2d-9159-e682dfdec09c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:39.938126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6b6c8-80e1-4f2d-9159-e682dfdec09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:39.974210 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-31c6b6c8-80e1-4f2d-9159-e682dfdec09c None None] MFA Rules not processed for user `b695d7a8c77b4250921140b894acae0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:39.975589 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-31c6b6c8-80e1-4f2d-9159-e682dfdec09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:40.064886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1421/2811] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:39 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:10:40.082721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9520a8-2ef9-419a-85d5-98b4782b69a3 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:40.084870 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5c9520a8-2ef9-419a-85d5-98b4782b69a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:40.182477 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9520a8-2ef9-419a-85d5-98b4782b69a3 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:40.189834 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e7a59-716c-4844-813b-380d141f4e1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:40.190803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e7a59-716c-4844-813b-380d141f4e1f service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:40.190803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e7a59-716c-4844-813b-380d141f4e1f service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:40.191794 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e7a59-716c-4844-813b-380d141f4e1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:40.255097 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e7a59-716c-4844-813b-380d141f4e1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:40.256795 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e7a59-716c-4844-813b-380d141f4e1f service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:40.270188 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1391/2812] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:10:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:10:50.061321 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=37 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:50.262231 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=61,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:10:58.923362 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53643994-a86f-4011-96a3-3c24dd586e55 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:58.923944 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53643994-a86f-4011-96a3-3c24dd586e55 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:58.925685 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53643994-a86f-4011-96a3-3c24dd586e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:58.944445 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-53643994-a86f-4011-96a3-3c24dd586e55 None None] MFA Rules not processed for user `c1e71f7596824b7e9c7a88a22a6664c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:10:58.946317 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-53643994-a86f-4011-96a3-3c24dd586e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:58.954605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1422/2813] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:10:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:10:58.996667 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e7a59-716c-4844-813b-380d141f4e1f service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:58.998210 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-247e7a59-716c-4844-813b-380d141f4e1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:59.057242 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e7a59-716c-4844-813b-380d141f4e1f service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:59.060892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc23d6-6d3f-4083-8b7a-83175d4f9d22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:59.061522 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc23d6-6d3f-4083-8b7a-83175d4f9d22 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:59.061803 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc23d6-6d3f-4083-8b7a-83175d4f9d22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:59.062109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc23d6-6d3f-4083-8b7a-83175d4f9d22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:59.063927 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d2cc23d6-6d3f-4083-8b7a-83175d4f9d22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:59.078712 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc23d6-6d3f-4083-8b7a-83175d4f9d22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:59.080166 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc23d6-6d3f-4083-8b7a-83175d4f9d22 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:59.096741 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1392/2814] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:10:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:10:59.399953 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53643994-a86f-4011-96a3-3c24dd586e55 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:10:59.401517 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-53643994-a86f-4011-96a3-3c24dd586e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:59.463551 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53643994-a86f-4011-96a3-3c24dd586e55 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:10:59.467490 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af000f27-e304-433b-bff0-d0f876417eaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:10:59.468119 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af000f27-e304-433b-bff0-d0f876417eaf service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:10:59.468397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af000f27-e304-433b-bff0-d0f876417eaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:10:59.468668 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af000f27-e304-433b-bff0-d0f876417eaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:10:59.470554 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-af000f27-e304-433b-bff0-d0f876417eaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:10:59.481797 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af000f27-e304-433b-bff0-d0f876417eaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:10:59.483268 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af000f27-e304-433b-bff0-d0f876417eaf service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:10:59.492876 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1423/2815] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:10:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:11:01.012890 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc23d6-6d3f-4083-8b7a-83175d4f9d22 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:11:01.023140 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc23d6-6d3f-4083-8b7a-83175d4f9d22 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:11:01.026828 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b808eb9-73fd-43c0-8dc2-8e542a7d7808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:11:01.027442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b808eb9-73fd-43c0-8dc2-8e542a7d7808 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:11:01.027728 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b808eb9-73fd-43c0-8dc2-8e542a7d7808 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:11:01.028006 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b808eb9-73fd-43c0-8dc2-8e542a7d7808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:11:01.029762 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2b808eb9-73fd-43c0-8dc2-8e542a7d7808 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:11:01.043937 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b808eb9-73fd-43c0-8dc2-8e542a7d7808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:11:01.045371 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b808eb9-73fd-43c0-8dc2-8e542a7d7808 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:11:01.059283 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1393/2816] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:11:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:11:09.487908 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=29 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:11:11.054071 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=32 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:11:14.930661 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af000f27-e304-433b-bff0-d0f876417eaf service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:11:14.939489 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af000f27-e304-433b-bff0-d0f876417eaf service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:11:14.943374 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fe241-f96c-40b1-8baa-48fa7469f996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:11:14.943981 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fe241-f96c-40b1-8baa-48fa7469f996 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:11:14.944881 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fe241-f96c-40b1-8baa-48fa7469f996 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:11:14.945156 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fe241-f96c-40b1-8baa-48fa7469f996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:11:14.946938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-341fe241-f96c-40b1-8baa-48fa7469f996 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:11:14.970468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fe241-f96c-40b1-8baa-48fa7469f996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:11:14.971905 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fe241-f96c-40b1-8baa-48fa7469f996 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:11:14.979940 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1424/2817] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:11:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:11:16.716227 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b808eb9-73fd-43c0-8dc2-8e542a7d7808 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:11:16.727295 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b808eb9-73fd-43c0-8dc2-8e542a7d7808 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:11:16.732287 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949356c-1b1c-47e3-a3b8-70b4d34ae10c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:11:16.732901 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949356c-1b1c-47e3-a3b8-70b4d34ae10c service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:11:16.733187 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949356c-1b1c-47e3-a3b8-70b4d34ae10c service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:11:16.733466 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949356c-1b1c-47e3-a3b8-70b4d34ae10c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:11:16.743459 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949356c-1b1c-47e3-a3b8-70b4d34ae10c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:11:16.745096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949356c-1b1c-47e3-a3b8-70b4d34ae10c service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:11:16.754751 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1394/2818] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:11:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:11:24.976295 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=11 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:11:26.750489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=6 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:12:16.587030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fe241-f96c-40b1-8baa-48fa7469f996 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:12:16.589012 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-341fe241-f96c-40b1-8baa-48fa7469f996 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:12:16.719499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fe241-f96c-40b1-8baa-48fa7469f996 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:12:16.724785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830c73b-f8ba-426d-bc38-5616d7c129db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:12:16.725417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830c73b-f8ba-426d-bc38-5616d7c129db service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:12:16.725700 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830c73b-f8ba-426d-bc38-5616d7c129db service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:12:16.726009 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830c73b-f8ba-426d-bc38-5616d7c129db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:12:16.727841 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9830c73b-f8ba-426d-bc38-5616d7c129db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:12:16.740151 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830c73b-f8ba-426d-bc38-5616d7c129db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:12:16.742027 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830c73b-f8ba-426d-bc38-5616d7c129db service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:12:16.752533 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1425/2819] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:12:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:12:26.748015 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=54 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:21.630495 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e8839-19c1-4524-9c82-0517d0c5a9d0 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:21.630999 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e8839-19c1-4524-9c82-0517d0c5a9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:21.631087 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e8839-19c1-4524-9c82-0517d0c5a9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:21.650775 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-239e8839-19c1-4524-9c82-0517d0c5a9d0 None None] MFA Rules not processed for user `c1e71f7596824b7e9c7a88a22a6664c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:21.651976 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-239e8839-19c1-4524-9c82-0517d0c5a9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:21.659558 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1395/2820] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:28.224442 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830c73b-f8ba-426d-bc38-5616d7c129db service placement] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:28.244193 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830c73b-f8ba-426d-bc38-5616d7c129db service placement] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:28.248946 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e2422-174b-4ffd-bd0b-7f0bfc3e4e1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:28.249562 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e2422-174b-4ffd-bd0b-7f0bfc3e4e1f service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:28.249942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e2422-174b-4ffd-bd0b-7f0bfc3e4e1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:28.250140 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e2422-174b-4ffd-bd0b-7f0bfc3e4e1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:28.251925 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b85e2422-174b-4ffd-bd0b-7f0bfc3e4e1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:28.318475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e2422-174b-4ffd-bd0b-7f0bfc3e4e1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:28.319941 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e2422-174b-4ffd-bd0b-7f0bfc3e4e1f service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:28.330496 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1426/2821] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:13:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:13:31.368258 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e8839-19c1-4524-9c82-0517d0c5a9d0 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:31.370221 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-239e8839-19c1-4524-9c82-0517d0c5a9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:31.427868 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e8839-19c1-4524-9c82-0517d0c5a9d0 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:31.430735 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243c684-115b-4a68-a438-7ce805fd032d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:31.431419 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243c684-115b-4a68-a438-7ce805fd032d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:31.431706 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243c684-115b-4a68-a438-7ce805fd032d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:31.431988 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243c684-115b-4a68-a438-7ce805fd032d admin admin] PATH_INFO: `/v3/users/ba149d1623e849328e399d6dbdca3e0c` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:31.433203 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243c684-115b-4a68-a438-7ce805fd032d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba149d1623e849328e399d6dbdca3e0c)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:31.435156 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243c684-115b-4a68-a438-7ce805fd032d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:31.484071 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3243c684-115b-4a68-a438-7ce805fd032d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba149d1623e849328e399d6dbdca3e0c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:31.484966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3243c684-115b-4a68-a438-7ce805fd032d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba149d1623e849328e399d6dbdca3e0c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:31.495012 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3243c684-115b-4a68-a438-7ce805fd032d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba149d1623e849328e399d6dbdca3e0c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:31.512170 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3243c684-115b-4a68-a438-7ce805fd032d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba149d1623e849328e399d6dbdca3e0c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:31.512787 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3243c684-115b-4a68-a438-7ce805fd032d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba149d1623e849328e399d6dbdca3e0c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:31.525978 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1396/2822] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:13:31 2022] DELETE /identity/v3/users/ba149d1623e849328e399d6dbdca3e0c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:31.704465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e2422-174b-4ffd-bd0b-7f0bfc3e4e1f service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:31.735273 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e2422-174b-4ffd-bd0b-7f0bfc3e4e1f service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:31.793673 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc85d9-3570-4885-9ead-e81ed3276545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:31.794251 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc85d9-3570-4885-9ead-e81ed3276545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:31.794527 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc85d9-3570-4885-9ead-e81ed3276545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:31.794796 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc85d9-3570-4885-9ead-e81ed3276545 admin admin] PATH_INFO: `/v3/projects/85354737a87146729a06bf22ddb3fefc` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:31.795938 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc85d9-3570-4885-9ead-e81ed3276545 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85354737a87146729a06bf22ddb3fefc)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:31.797816 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc85d9-3570-4885-9ead-e81ed3276545 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:31.842662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-10dc85d9-3570-4885-9ead-e81ed3276545 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85354737a87146729a06bf22ddb3fefc', 'request_id': 'req-10dc85d9-3570-4885-9ead-e81ed3276545'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:31.848751 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-10dc85d9-3570-4885-9ead-e81ed3276545 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85354737a87146729a06bf22ddb3fefc', 'request_id': 'req-10dc85d9-3570-4885-9ead-e81ed3276545'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:31.855183 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-10dc85d9-3570-4885-9ead-e81ed3276545 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85354737a87146729a06bf22ddb3fefc', 'request_id': 'req-10dc85d9-3570-4885-9ead-e81ed3276545'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:31.870115 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:31.877771 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-10dc85d9-3570-4885-9ead-e81ed3276545 admin admin] The token cache is being invalidate because project 85354737a87146729a06bf22ddb3fefc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:13:31.878463 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-10dc85d9-3570-4885-9ead-e81ed3276545 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:31.879352 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-10dc85d9-3570-4885-9ead-e81ed3276545 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:31.880974 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1427/2823] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:13:31 2022] DELETE /identity/v3/projects/85354737a87146729a06bf22ddb3fefc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:31.888360 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447f4a7-7982-4c11-abfd-b4e55dc4e745 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:31.888655 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447f4a7-7982-4c11-abfd-b4e55dc4e745 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:31.888934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447f4a7-7982-4c11-abfd-b4e55dc4e745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:31.920628 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d447f4a7-7982-4c11-abfd-b4e55dc4e745 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:31.921856 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d447f4a7-7982-4c11-abfd-b4e55dc4e745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:31.989998 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1397/2824] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:31.998037 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88f589-a4b4-45a8-bd94-dd26c5520b81 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:31.998326 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88f589-a4b4-45a8-bd94-dd26c5520b81 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:31.999106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88f589-a4b4-45a8-bd94-dd26c5520b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:32.019817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-fa88f589-a4b4-45a8-bd94-dd26c5520b81 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:32.021075 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fa88f589-a4b4-45a8-bd94-dd26c5520b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:32.029895 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1428/2825] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:13:32.035203 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447f4a7-7982-4c11-abfd-b4e55dc4e745 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:32.045030 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447f4a7-7982-4c11-abfd-b4e55dc4e745 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:32.050882 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9d272-f8a7-4f4a-a01c-2b7e0bbd2c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:32.051675 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9d272-f8a7-4f4a-a01c-2b7e0bbd2c14 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:32.051958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9d272-f8a7-4f4a-a01c-2b7e0bbd2c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:32.052320 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9d272-f8a7-4f4a-a01c-2b7e0bbd2c14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:32.053049 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9d272-f8a7-4f4a-a01c-2b7e0bbd2c14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:13:32.053432 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9d272-f8a7-4f4a-a01c-2b7e0bbd2c14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:32.054966 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9d272-f8a7-4f4a-a01c-2b7e0bbd2c14 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:32.071571 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1398/2826] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:13:32.076370 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88f589-a4b4-45a8-bd94-dd26c5520b81 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:32.085687 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88f589-a4b4-45a8-bd94-dd26c5520b81 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:32.090407 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea69605-0451-4c37-9209-83e626d754c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:32.091018 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea69605-0451-4c37-9209-83e626d754c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:32.091295 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea69605-0451-4c37-9209-83e626d754c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:32.091559 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea69605-0451-4c37-9209-83e626d754c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:32.092217 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea69605-0451-4c37-9209-83e626d754c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:32.094126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea69605-0451-4c37-9209-83e626d754c9 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:32.122618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1429/2827] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:13:32 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:13:32.126853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9d272-f8a7-4f4a-a01c-2b7e0bbd2c14 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:32.137099 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9d272-f8a7-4f4a-a01c-2b7e0bbd2c14 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:32.154284 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:32.195269 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce42557-9ca9-4940-b8b7-a6dda8ed1d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:32.195907 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce42557-9ca9-4940-b8b7-a6dda8ed1d34 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:32.196194 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce42557-9ca9-4940-b8b7-a6dda8ed1d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:32.196472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce42557-9ca9-4940-b8b7-a6dda8ed1d34 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:32.197155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce42557-9ca9-4940-b8b7-a6dda8ed1d34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:32.199136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce42557-9ca9-4940-b8b7-a6dda8ed1d34 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:32.221372 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1399/2828] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:13:32 2022] POST /identity/v3/users => generated 282 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:13:32.226799 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea69605-0451-4c37-9209-83e626d754c9 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:32.237226 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea69605-0451-4c37-9209-83e626d754c9 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:32.242004 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb40db-afdb-49b6-8f7a-a1d106bb0404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:32.242861 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb40db-afdb-49b6-8f7a-a1d106bb0404 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:32.243149 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb40db-afdb-49b6-8f7a-a1d106bb0404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:32.243422 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb40db-afdb-49b6-8f7a-a1d106bb0404 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:32.244111 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb40db-afdb-49b6-8f7a-a1d106bb0404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:32.245698 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb40db-afdb-49b6-8f7a-a1d106bb0404 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:32.256971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1430/2829] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:13:32.261134 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce42557-9ca9-4940-b8b7-a6dda8ed1d34 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:32.271538 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce42557-9ca9-4940-b8b7-a6dda8ed1d34 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:32.276129 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269af83-53f9-4c7e-a0a7-91dedacb5c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:32.276731 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269af83-53f9-4c7e-a0a7-91dedacb5c2f admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:32.277010 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269af83-53f9-4c7e-a0a7-91dedacb5c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:32.277282 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269af83-53f9-4c7e-a0a7-91dedacb5c2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:32.278009 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269af83-53f9-4c7e-a0a7-91dedacb5c2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:32.279599 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269af83-53f9-4c7e-a0a7-91dedacb5c2f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:32.290736 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1400/2830] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:13:32.295438 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb40db-afdb-49b6-8f7a-a1d106bb0404 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:32.305740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb40db-afdb-49b6-8f7a-a1d106bb0404 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:32.310359 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa9b5a-55d2-4be5-8801-de85d92777a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:32.311027 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa9b5a-55d2-4be5-8801-de85d92777a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:32.311305 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa9b5a-55d2-4be5-8801-de85d92777a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:32.311576 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa9b5a-55d2-4be5-8801-de85d92777a7 admin admin] PATH_INFO: `/v3/projects/62b9750d57d94ed7b4c3faf482f13e55/users/77644f8de1714277ad6d400933788814/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:32.326475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa9b5a-55d2-4be5-8801-de85d92777a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62b9750d57d94ed7b4c3faf482f13e55, user_id=77644f8de1714277ad6d400933788814, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:32.329072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa9b5a-55d2-4be5-8801-de85d92777a7 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:32.342330 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1431/2831] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:13:32 2022] PUT /identity/v3/projects/62b9750d57d94ed7b4c3faf482f13e55/users/77644f8de1714277ad6d400933788814/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:32.410916 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5202958-2ca1-41a3-b659-c3e96eb38fd9 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:32.411477 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5202958-2ca1-41a3-b659-c3e96eb38fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:32.411477 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5202958-2ca1-41a3-b659-c3e96eb38fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:32.439591 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e5202958-2ca1-41a3-b659-c3e96eb38fd9 None None] MFA Rules not processed for user `77644f8de1714277ad6d400933788814`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:32.440821 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e5202958-2ca1-41a3-b659-c3e96eb38fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:32.504454 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1401/2832] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:32 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:32.509090 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa9b5a-55d2-4be5-8801-de85d92777a7 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:32.574626 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa9b5a-55d2-4be5-8801-de85d92777a7 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:32.579276 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003b28e-4a55-48f7-aa0b-1568371e7545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:32.579894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003b28e-4a55-48f7-aa0b-1568371e7545 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:32.580174 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003b28e-4a55-48f7-aa0b-1568371e7545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:32.580453 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003b28e-4a55-48f7-aa0b-1568371e7545 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:32.581671 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003b28e-4a55-48f7-aa0b-1568371e7545 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:32.583594 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003b28e-4a55-48f7-aa0b-1568371e7545 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:32.610790 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1432/2833] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:13:32 2022] POST /identity/v3/projects => generated 361 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:13:32.614965 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5202958-2ca1-41a3-b659-c3e96eb38fd9 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:32.625761 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5202958-2ca1-41a3-b659-c3e96eb38fd9 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:32.682524 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1b1cb-8427-4c32-acf8-ec298c6b57b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:32.683152 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1b1cb-8427-4c32-acf8-ec298c6b57b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:32.683439 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1b1cb-8427-4c32-acf8-ec298c6b57b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:32.683715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1b1cb-8427-4c32-acf8-ec298c6b57b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:32.684375 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1b1cb-8427-4c32-acf8-ec298c6b57b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:32.686332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1b1cb-8427-4c32-acf8-ec298c6b57b1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:32.709714 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1402/2834] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:13:32 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:13:32.714396 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003b28e-4a55-48f7-aa0b-1568371e7545 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:32.724840 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003b28e-4a55-48f7-aa0b-1568371e7545 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:32.729663 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83a78f-33bd-4535-87f8-ce43d8862f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:32.730319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83a78f-33bd-4535-87f8-ce43d8862f81 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:32.730607 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83a78f-33bd-4535-87f8-ce43d8862f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:32.730885 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83a78f-33bd-4535-87f8-ce43d8862f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:32.731575 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83a78f-33bd-4535-87f8-ce43d8862f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:32.733169 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83a78f-33bd-4535-87f8-ce43d8862f81 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:32.744681 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1433/2835] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:13:32.749069 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1b1cb-8427-4c32-acf8-ec298c6b57b1 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:32.760465 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1b1cb-8427-4c32-acf8-ec298c6b57b1 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:32.765451 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a2eba-c99b-40bc-9bc0-da2ff5f0bf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:32.766174 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a2eba-c99b-40bc-9bc0-da2ff5f0bf3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:32.766460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a2eba-c99b-40bc-9bc0-da2ff5f0bf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:32.766739 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a2eba-c99b-40bc-9bc0-da2ff5f0bf3e admin admin] PATH_INFO: `/v3/projects/231fdb03c1694fd39e3f148f2e2d74c3/users/ab1dd229b2f9441f9bf56845bab14440/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:32.782336 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a2eba-c99b-40bc-9bc0-da2ff5f0bf3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=231fdb03c1694fd39e3f148f2e2d74c3, user_id=ab1dd229b2f9441f9bf56845bab14440, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:32.785009 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a2eba-c99b-40bc-9bc0-da2ff5f0bf3e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:32.787251 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:32.799455 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1403/2836] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:13:32 2022] PUT /identity/v3/projects/231fdb03c1694fd39e3f148f2e2d74c3/users/ab1dd229b2f9441f9bf56845bab14440/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:32.803647 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83a78f-33bd-4535-87f8-ce43d8862f81 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:32.813685 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83a78f-33bd-4535-87f8-ce43d8862f81 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:32.870673 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b755f4-9305-4ea2-b3b8-57e25581c604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:32.871286 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b755f4-9305-4ea2-b3b8-57e25581c604 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:32.871564 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b755f4-9305-4ea2-b3b8-57e25581c604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:32.871837 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b755f4-9305-4ea2-b3b8-57e25581c604 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:32.872511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b755f4-9305-4ea2-b3b8-57e25581c604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:32.874110 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b755f4-9305-4ea2-b3b8-57e25581c604 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:32.884284 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1434/2837] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:13:32.888442 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a2eba-c99b-40bc-9bc0-da2ff5f0bf3e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:32.899580 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a2eba-c99b-40bc-9bc0-da2ff5f0bf3e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:32.904526 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfbc2aa-cdbe-4f21-b512-15a682603fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:32.905205 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbc2aa-cdbe-4f21-b512-15a682603fa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:32.905490 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbc2aa-cdbe-4f21-b512-15a682603fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:32.905767 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbc2aa-cdbe-4f21-b512-15a682603fa8 admin admin] PATH_INFO: `/v3/projects/231fdb03c1694fd39e3f148f2e2d74c3/users/ab1dd229b2f9441f9bf56845bab14440/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:32.907780 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfbc2aa-cdbe-4f21-b512-15a682603fa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=231fdb03c1694fd39e3f148f2e2d74c3, user_id=ab1dd229b2f9441f9bf56845bab14440, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:32.910707 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfbc2aa-cdbe-4f21-b512-15a682603fa8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:32.924472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1404/2838] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:13:32 2022] PUT /identity/v3/projects/231fdb03c1694fd39e3f148f2e2d74c3/users/ab1dd229b2f9441f9bf56845bab14440/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:32.934803 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e7706-5f60-4092-85b8-9c0c5413089f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:32.935087 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e7706-5f60-4092-85b8-9c0c5413089f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:32.935361 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e7706-5f60-4092-85b8-9c0c5413089f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:32.944240 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:32.965190 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-109e7706-5f60-4092-85b8-9c0c5413089f None None] MFA Rules not processed for user `ab1dd229b2f9441f9bf56845bab14440`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:32.966871 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-109e7706-5f60-4092-85b8-9c0c5413089f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:33.034481 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1435/2839] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:32 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:33.046148 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfbc2aa-cdbe-4f21-b512-15a682603fa8 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:33.048177 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9bfbc2aa-cdbe-4f21-b512-15a682603fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:33.125330 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfbc2aa-cdbe-4f21-b512-15a682603fa8 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:33.132634 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a320e-7b7d-48e1-a13d-efa9f352f216 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:33.133278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a320e-7b7d-48e1-a13d-efa9f352f216 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:33.133563 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a320e-7b7d-48e1-a13d-efa9f352f216 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:33.133841 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a320e-7b7d-48e1-a13d-efa9f352f216 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:33.144324 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a320e-7b7d-48e1-a13d-efa9f352f216 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:33.145550 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a320e-7b7d-48e1-a13d-efa9f352f216 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:33.156852 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1405/2840] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:13:33 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:13:33.251260 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e7706-5f60-4092-85b8-9c0c5413089f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:33.264435 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e7706-5f60-4092-85b8-9c0c5413089f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:33.268092 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af8044-7e0e-4954-a874-56a8089e7f66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:33.268860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af8044-7e0e-4954-a874-56a8089e7f66 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:33.269147 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af8044-7e0e-4954-a874-56a8089e7f66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:33.269447 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af8044-7e0e-4954-a874-56a8089e7f66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:33.337773 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af8044-7e0e-4954-a874-56a8089e7f66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:33.339250 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af8044-7e0e-4954-a874-56a8089e7f66 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:33.350070 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1436/2841] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:13:33 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:13:36.092860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a320e-7b7d-48e1-a13d-efa9f352f216 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:36.156859 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a320e-7b7d-48e1-a13d-efa9f352f216 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:36.161390 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570acfcb-c55f-4ab6-b539-a29f68eb78b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:36.162062 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570acfcb-c55f-4ab6-b539-a29f68eb78b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:36.162349 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570acfcb-c55f-4ab6-b539-a29f68eb78b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:36.162625 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570acfcb-c55f-4ab6-b539-a29f68eb78b2 admin admin] PATH_INFO: `/v3/users/77644f8de1714277ad6d400933788814` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:36.163786 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570acfcb-c55f-4ab6-b539-a29f68eb78b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77644f8de1714277ad6d400933788814)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:36.165702 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570acfcb-c55f-4ab6-b539-a29f68eb78b2 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:36.213276 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-570acfcb-c55f-4ab6-b539-a29f68eb78b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77644f8de1714277ad6d400933788814'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:36.213959 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-570acfcb-c55f-4ab6-b539-a29f68eb78b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77644f8de1714277ad6d400933788814'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:36.222151 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-570acfcb-c55f-4ab6-b539-a29f68eb78b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77644f8de1714277ad6d400933788814'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:36.234521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-570acfcb-c55f-4ab6-b539-a29f68eb78b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77644f8de1714277ad6d400933788814'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:36.235160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-570acfcb-c55f-4ab6-b539-a29f68eb78b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77644f8de1714277ad6d400933788814'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:36.240306 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:36.251409 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1406/2842] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:13:36 2022] DELETE /identity/v3/users/77644f8de1714277ad6d400933788814 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:36.255949 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86a2dd-2871-49c1-874e-21b1d50978e8 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:36.256402 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86a2dd-2871-49c1-874e-21b1d50978e8 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:36.256513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86a2dd-2871-49c1-874e-21b1d50978e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:36.275545 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1e86a2dd-2871-49c1-874e-21b1d50978e8 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:36.276749 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1e86a2dd-2871-49c1-874e-21b1d50978e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:36.335624 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1437/2843] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:13:36.345501 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570acfcb-c55f-4ab6-b539-a29f68eb78b2 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:36.346977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-570acfcb-c55f-4ab6-b539-a29f68eb78b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:36.404345 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570acfcb-c55f-4ab6-b539-a29f68eb78b2 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:36.408026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1aac3-2d9d-474e-9aa4-9a591c3e7196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:36.408643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1aac3-2d9d-474e-9aa4-9a591c3e7196 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:36.408924 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1aac3-2d9d-474e-9aa4-9a591c3e7196 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:36.409197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1aac3-2d9d-474e-9aa4-9a591c3e7196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:36.419936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1aac3-2d9d-474e-9aa4-9a591c3e7196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:36.421520 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1aac3-2d9d-474e-9aa4-9a591c3e7196 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:36.432106 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1407/2844] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:13:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:13:36.705057 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86a2dd-2871-49c1-874e-21b1d50978e8 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:36.706502 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1e86a2dd-2871-49c1-874e-21b1d50978e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:36.720954 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86a2dd-2871-49c1-874e-21b1d50978e8 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:36.726582 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d79306-6d11-49c9-8706-09608483d4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:36.727106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d79306-6d11-49c9-8706-09608483d4e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:36.727417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d79306-6d11-49c9-8706-09608483d4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:36.727664 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d79306-6d11-49c9-8706-09608483d4e4 admin admin] PATH_INFO: `/v3/projects/62b9750d57d94ed7b4c3faf482f13e55` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:36.728850 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d79306-6d11-49c9-8706-09608483d4e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62b9750d57d94ed7b4c3faf482f13e55)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:36.730816 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d79306-6d11-49c9-8706-09608483d4e4 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:36.773297 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:36.798167 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-33d79306-6d11-49c9-8706-09608483d4e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62b9750d57d94ed7b4c3faf482f13e55', 'request_id': 'req-33d79306-6d11-49c9-8706-09608483d4e4'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:36.804054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-33d79306-6d11-49c9-8706-09608483d4e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62b9750d57d94ed7b4c3faf482f13e55', 'request_id': 'req-33d79306-6d11-49c9-8706-09608483d4e4'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:36.811061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-33d79306-6d11-49c9-8706-09608483d4e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62b9750d57d94ed7b4c3faf482f13e55', 'request_id': 'req-33d79306-6d11-49c9-8706-09608483d4e4'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:36.827924 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-33d79306-6d11-49c9-8706-09608483d4e4 admin admin] The token cache is being invalidate because project 62b9750d57d94ed7b4c3faf482f13e55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:13:36.828218 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-33d79306-6d11-49c9-8706-09608483d4e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:36.829032 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-33d79306-6d11-49c9-8706-09608483d4e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:36.830638 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1438/2845] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:13:36 2022] DELETE /identity/v3/projects/62b9750d57d94ed7b4c3faf482f13e55 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:36.835178 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1aac3-2d9d-474e-9aa4-9a591c3e7196 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:36.846353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1aac3-2d9d-474e-9aa4-9a591c3e7196 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:36.908373 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb1178-7ba4-4d01-92fe-215ada0f6cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:36.909123 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb1178-7ba4-4d01-92fe-215ada0f6cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:36.909407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb1178-7ba4-4d01-92fe-215ada0f6cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:36.909685 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb1178-7ba4-4d01-92fe-215ada0f6cba admin admin] PATH_INFO: `/v3/users/ab1dd229b2f9441f9bf56845bab14440` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:36.911468 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb1178-7ba4-4d01-92fe-215ada0f6cba admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab1dd229b2f9441f9bf56845bab14440)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:36.913391 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb1178-7ba4-4d01-92fe-215ada0f6cba admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:36.963862 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cdeb1178-7ba4-4d01-92fe-215ada0f6cba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab1dd229b2f9441f9bf56845bab14440'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:36.964545 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cdeb1178-7ba4-4d01-92fe-215ada0f6cba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab1dd229b2f9441f9bf56845bab14440'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:36.972022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cdeb1178-7ba4-4d01-92fe-215ada0f6cba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab1dd229b2f9441f9bf56845bab14440'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:36.985838 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cdeb1178-7ba4-4d01-92fe-215ada0f6cba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab1dd229b2f9441f9bf56845bab14440'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:36.986548 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cdeb1178-7ba4-4d01-92fe-215ada0f6cba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab1dd229b2f9441f9bf56845bab14440'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:36.998967 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1408/2846] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:13:36 2022] DELETE /identity/v3/users/ab1dd229b2f9441f9bf56845bab14440 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:37.283792 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d79306-6d11-49c9-8706-09608483d4e4 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:37.285343 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-33d79306-6d11-49c9-8706-09608483d4e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:37.303209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d79306-6d11-49c9-8706-09608483d4e4 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:37.369495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b3b9f-cc17-4b01-8f53-4e7df386bb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:37.370178 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b3b9f-cc17-4b01-8f53-4e7df386bb9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:37.370463 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b3b9f-cc17-4b01-8f53-4e7df386bb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:37.370780 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b3b9f-cc17-4b01-8f53-4e7df386bb9e admin admin] PATH_INFO: `/v3/projects/231fdb03c1694fd39e3f148f2e2d74c3` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:37.371963 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b3b9f-cc17-4b01-8f53-4e7df386bb9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=231fdb03c1694fd39e3f148f2e2d74c3)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:37.373842 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b3b9f-cc17-4b01-8f53-4e7df386bb9e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:37.421831 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-cd7b3b9f-cc17-4b01-8f53-4e7df386bb9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '231fdb03c1694fd39e3f148f2e2d74c3', 'request_id': 'req-cd7b3b9f-cc17-4b01-8f53-4e7df386bb9e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:37.428353 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-cd7b3b9f-cc17-4b01-8f53-4e7df386bb9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '231fdb03c1694fd39e3f148f2e2d74c3', 'request_id': 'req-cd7b3b9f-cc17-4b01-8f53-4e7df386bb9e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:37.434456 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-cd7b3b9f-cc17-4b01-8f53-4e7df386bb9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '231fdb03c1694fd39e3f148f2e2d74c3', 'request_id': 'req-cd7b3b9f-cc17-4b01-8f53-4e7df386bb9e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:37.449237 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-cd7b3b9f-cc17-4b01-8f53-4e7df386bb9e admin admin] The token cache is being invalidate because project 231fdb03c1694fd39e3f148f2e2d74c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:13:37.449532 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-cd7b3b9f-cc17-4b01-8f53-4e7df386bb9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:37.450215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-cd7b3b9f-cc17-4b01-8f53-4e7df386bb9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:37.451763 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1439/2847] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:13:37 2022] DELETE /identity/v3/projects/231fdb03c1694fd39e3f148f2e2d74c3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:37.456681 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d6ba2-9598-488e-be3e-1919ab45859c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:37.457204 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d6ba2-9598-488e-be3e-1919ab45859c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:37.457295 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d6ba2-9598-488e-be3e-1919ab45859c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:37.477551 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:37.509273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3e8d6ba2-9598-488e-be3e-1919ab45859c None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:37.510462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3e8d6ba2-9598-488e-be3e-1919ab45859c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:37.569193 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1409/2848] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:37.577875 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a7aa9-fabc-4ff5-9a2f-a1a3fdbc90a3 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:37.578184 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a7aa9-fabc-4ff5-9a2f-a1a3fdbc90a3 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:37.578454 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a7aa9-fabc-4ff5-9a2f-a1a3fdbc90a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:37.597484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-bb0a7aa9-fabc-4ff5-9a2f-a1a3fdbc90a3 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:37.598699 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-bb0a7aa9-fabc-4ff5-9a2f-a1a3fdbc90a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:37.606666 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1440/2849] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:37.610842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d6ba2-9598-488e-be3e-1919ab45859c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:37.621473 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d6ba2-9598-488e-be3e-1919ab45859c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:37.626095 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801d55f-d9d5-45f0-a5e6-82be7035fc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:37.626704 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801d55f-d9d5-45f0-a5e6-82be7035fc33 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:37.627774 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801d55f-d9d5-45f0-a5e6-82be7035fc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:37.627774 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801d55f-d9d5-45f0-a5e6-82be7035fc33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:37.627995 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801d55f-d9d5-45f0-a5e6-82be7035fc33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:13:37.628259 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801d55f-d9d5-45f0-a5e6-82be7035fc33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:37.630520 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801d55f-d9d5-45f0-a5e6-82be7035fc33 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:37.643472 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1410/2850] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:13:37.648105 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a7aa9-fabc-4ff5-9a2f-a1a3fdbc90a3 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:37.657515 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a7aa9-fabc-4ff5-9a2f-a1a3fdbc90a3 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:37.662170 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2d689-d1d9-4642-a37b-fce4e90d2822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:37.662781 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2d689-d1d9-4642-a37b-fce4e90d2822 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:37.663064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2d689-d1d9-4642-a37b-fce4e90d2822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:37.663337 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2d689-d1d9-4642-a37b-fce4e90d2822 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:37.664002 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2d689-d1d9-4642-a37b-fce4e90d2822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:37.665880 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2d689-d1d9-4642-a37b-fce4e90d2822 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:37.703479 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1441/2851] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:13:37 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:13:37.707462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801d55f-d9d5-45f0-a5e6-82be7035fc33 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:37.717872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801d55f-d9d5-45f0-a5e6-82be7035fc33 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:37.774382 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57448e62-f3f1-4de9-8cd2-32e2b9fe83b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:37.775079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57448e62-f3f1-4de9-8cd2-32e2b9fe83b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:37.775259 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57448e62-f3f1-4de9-8cd2-32e2b9fe83b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:37.775460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57448e62-f3f1-4de9-8cd2-32e2b9fe83b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:37.776136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57448e62-f3f1-4de9-8cd2-32e2b9fe83b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:37.778086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57448e62-f3f1-4de9-8cd2-32e2b9fe83b9 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:37.802307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1411/2852] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:13:37 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:13:37.806955 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2d689-d1d9-4642-a37b-fce4e90d2822 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:37.813734 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:37.817964 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2d689-d1d9-4642-a37b-fce4e90d2822 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:37.824425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a1c4a-99c4-4b52-9f61-2aec491ce466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:37.825049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a1c4a-99c4-4b52-9f61-2aec491ce466 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:37.825495 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a1c4a-99c4-4b52-9f61-2aec491ce466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:37.825843 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a1c4a-99c4-4b52-9f61-2aec491ce466 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:37.826802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a1c4a-99c4-4b52-9f61-2aec491ce466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:37.828393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a1c4a-99c4-4b52-9f61-2aec491ce466 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:37.838755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1442/2853] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:13:37.843079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57448e62-f3f1-4de9-8cd2-32e2b9fe83b9 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:37.853446 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57448e62-f3f1-4de9-8cd2-32e2b9fe83b9 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:37.857997 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c594961-a94f-45b2-93ea-8d991e66cd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:37.858600 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c594961-a94f-45b2-93ea-8d991e66cd34 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:37.858884 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c594961-a94f-45b2-93ea-8d991e66cd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:37.859154 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c594961-a94f-45b2-93ea-8d991e66cd34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:37.859837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c594961-a94f-45b2-93ea-8d991e66cd34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:37.861460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c594961-a94f-45b2-93ea-8d991e66cd34 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:37.873467 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1412/2854] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:13:37.878395 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a1c4a-99c4-4b52-9f61-2aec491ce466 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:37.895942 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a1c4a-99c4-4b52-9f61-2aec491ce466 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:37.900700 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09b6cf-981f-4948-b890-aeb1533bd96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:37.901406 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09b6cf-981f-4948-b890-aeb1533bd96c admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:37.901716 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09b6cf-981f-4948-b890-aeb1533bd96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:37.902228 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09b6cf-981f-4948-b890-aeb1533bd96c admin admin] PATH_INFO: `/v3/projects/81959ca275bd462fb319bc6a82f3cfdd/users/910a9ffd40364d04b8789afb96496989/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:37.918172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc09b6cf-981f-4948-b890-aeb1533bd96c admin admin] RBAC: Authorizing `identity:create_grant(project_id=81959ca275bd462fb319bc6a82f3cfdd, user_id=910a9ffd40364d04b8789afb96496989, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:37.921004 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc09b6cf-981f-4948-b890-aeb1533bd96c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:37.942465 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1443/2855] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:13:37 2022] PUT /identity/v3/projects/81959ca275bd462fb319bc6a82f3cfdd/users/910a9ffd40364d04b8789afb96496989/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:37.952309 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06b551-3f0e-42fe-bb27-237443853da7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:37.952603 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06b551-3f0e-42fe-bb27-237443853da7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:37.952854 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06b551-3f0e-42fe-bb27-237443853da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:37.981789 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-fe06b551-3f0e-42fe-bb27-237443853da7 None None] MFA Rules not processed for user `910a9ffd40364d04b8789afb96496989`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:37.983084 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-fe06b551-3f0e-42fe-bb27-237443853da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:38.028586 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:38.057050 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1413/2856] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:37 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:38.061618 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09b6cf-981f-4948-b890-aeb1533bd96c admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:38.135970 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09b6cf-981f-4948-b890-aeb1533bd96c admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:38.143064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce3c49-b8e7-4eb8-a468-2aa1552e25fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:38.143693 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce3c49-b8e7-4eb8-a468-2aa1552e25fa admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:38.143977 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce3c49-b8e7-4eb8-a468-2aa1552e25fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:38.144820 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce3c49-b8e7-4eb8-a468-2aa1552e25fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:38.145497 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ce3c49-b8e7-4eb8-a468-2aa1552e25fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:38.147461 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ce3c49-b8e7-4eb8-a468-2aa1552e25fa admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:38.178888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1444/2857] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:13:38 2022] POST /identity/v3/projects => generated 367 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:13:38.183165 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06b551-3f0e-42fe-bb27-237443853da7 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:38.194764 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06b551-3f0e-42fe-bb27-237443853da7 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:38.257392 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5aaa3-1cb3-4eff-89e1-1fb86e626bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:38.258060 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5aaa3-1cb3-4eff-89e1-1fb86e626bbc admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:38.258339 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5aaa3-1cb3-4eff-89e1-1fb86e626bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:38.258611 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5aaa3-1cb3-4eff-89e1-1fb86e626bbc admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:38.259273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5aaa3-1cb3-4eff-89e1-1fb86e626bbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:38.261164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5aaa3-1cb3-4eff-89e1-1fb86e626bbc admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:38.286530 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1414/2858] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:13:38 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:13:38.290430 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce3c49-b8e7-4eb8-a468-2aa1552e25fa admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:38.301017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce3c49-b8e7-4eb8-a468-2aa1552e25fa admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:38.306594 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4474ea-81d9-432a-b3c4-6cffafdf5486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:38.307231 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4474ea-81d9-432a-b3c4-6cffafdf5486 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:38.307516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4474ea-81d9-432a-b3c4-6cffafdf5486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:38.307794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4474ea-81d9-432a-b3c4-6cffafdf5486 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:38.308496 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4474ea-81d9-432a-b3c4-6cffafdf5486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:38.310188 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4474ea-81d9-432a-b3c4-6cffafdf5486 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:38.322159 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1445/2859] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:13:38.326378 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5aaa3-1cb3-4eff-89e1-1fb86e626bbc admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:38.338421 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5aaa3-1cb3-4eff-89e1-1fb86e626bbc admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:38.343837 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0175c6-fe19-46d0-9fb7-87efc889b99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:38.344518 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0175c6-fe19-46d0-9fb7-87efc889b99a admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:38.344804 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0175c6-fe19-46d0-9fb7-87efc889b99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:38.345083 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0175c6-fe19-46d0-9fb7-87efc889b99a admin admin] PATH_INFO: `/v3/projects/e604fd3316e24d488cedcca1605b3393/users/6e9c7d72d8d24e90ab8ba8902119782f/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:38.361101 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0175c6-fe19-46d0-9fb7-87efc889b99a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e604fd3316e24d488cedcca1605b3393, user_id=6e9c7d72d8d24e90ab8ba8902119782f, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:38.364071 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0175c6-fe19-46d0-9fb7-87efc889b99a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:38.377942 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1415/2860] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:13:38 2022] PUT /identity/v3/projects/e604fd3316e24d488cedcca1605b3393/users/6e9c7d72d8d24e90ab8ba8902119782f/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:38.383804 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4474ea-81d9-432a-b3c4-6cffafdf5486 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:38.393985 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4474ea-81d9-432a-b3c4-6cffafdf5486 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:38.454629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfde967-6a62-4528-b621-8b5b61e9cacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:38.455252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfde967-6a62-4528-b621-8b5b61e9cacc admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:38.455540 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfde967-6a62-4528-b621-8b5b61e9cacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:38.455813 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfde967-6a62-4528-b621-8b5b61e9cacc admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:38.456514 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfde967-6a62-4528-b621-8b5b61e9cacc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:38.458657 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfde967-6a62-4528-b621-8b5b61e9cacc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:38.468601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1446/2861] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:13:38.473443 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0175c6-fe19-46d0-9fb7-87efc889b99a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:38.483648 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0175c6-fe19-46d0-9fb7-87efc889b99a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:38.488479 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c736b512-11a0-4384-95bf-b2ca3268c90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:38.489197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736b512-11a0-4384-95bf-b2ca3268c90d admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:38.489489 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736b512-11a0-4384-95bf-b2ca3268c90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:38.489807 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736b512-11a0-4384-95bf-b2ca3268c90d admin admin] PATH_INFO: `/v3/projects/e604fd3316e24d488cedcca1605b3393/users/6e9c7d72d8d24e90ab8ba8902119782f/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:38.491856 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c736b512-11a0-4384-95bf-b2ca3268c90d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e604fd3316e24d488cedcca1605b3393, user_id=6e9c7d72d8d24e90ab8ba8902119782f, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:38.494815 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c736b512-11a0-4384-95bf-b2ca3268c90d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:38.506244 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1416/2862] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:13:38 2022] PUT /identity/v3/projects/e604fd3316e24d488cedcca1605b3393/users/6e9c7d72d8d24e90ab8ba8902119782f/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:13:38.516613 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154d5ef-2835-4acb-8dd0-84e252703dab None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:38.516906 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154d5ef-2835-4acb-8dd0-84e252703dab None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:38.517183 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154d5ef-2835-4acb-8dd0-84e252703dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:38.546634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-3154d5ef-2835-4acb-8dd0-84e252703dab None None] MFA Rules not processed for user `6e9c7d72d8d24e90ab8ba8902119782f`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:38.547867 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3154d5ef-2835-4acb-8dd0-84e252703dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:38.579669 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:38.624211 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1447/2863] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:38 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:38.636257 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c736b512-11a0-4384-95bf-b2ca3268c90d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:38.638304 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c736b512-11a0-4384-95bf-b2ca3268c90d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:38.700745 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c736b512-11a0-4384-95bf-b2ca3268c90d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:38.707938 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108756f-0522-45ac-8683-21ebe5d417c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:38.708780 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108756f-0522-45ac-8683-21ebe5d417c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:38.709202 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108756f-0522-45ac-8683-21ebe5d417c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:38.709595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108756f-0522-45ac-8683-21ebe5d417c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:38.719774 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108756f-0522-45ac-8683-21ebe5d417c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:38.721253 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108756f-0522-45ac-8683-21ebe5d417c1 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:38.731553 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1417/2864] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:13:38 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:13:40.199387 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154d5ef-2835-4acb-8dd0-84e252703dab None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:40.281207 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154d5ef-2835-4acb-8dd0-84e252703dab None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:40.287133 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891e80b-b269-4ff4-b89b-67be9e012480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:40.287877 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891e80b-b269-4ff4-b89b-67be9e012480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:40.288779 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891e80b-b269-4ff4-b89b-67be9e012480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:40.289064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891e80b-b269-4ff4-b89b-67be9e012480 admin admin] PATH_INFO: `/v3/users/910a9ffd40364d04b8789afb96496989` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:40.290436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891e80b-b269-4ff4-b89b-67be9e012480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=910a9ffd40364d04b8789afb96496989)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:40.293536 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891e80b-b269-4ff4-b89b-67be9e012480 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:40.361812 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d891e80b-b269-4ff4-b89b-67be9e012480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '910a9ffd40364d04b8789afb96496989'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:40.362429 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d891e80b-b269-4ff4-b89b-67be9e012480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '910a9ffd40364d04b8789afb96496989'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:40.371608 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d891e80b-b269-4ff4-b89b-67be9e012480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '910a9ffd40364d04b8789afb96496989'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:40.388785 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d891e80b-b269-4ff4-b89b-67be9e012480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '910a9ffd40364d04b8789afb96496989'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:40.397292 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d891e80b-b269-4ff4-b89b-67be9e012480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '910a9ffd40364d04b8789afb96496989'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:40.403930 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1448/2865] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:13:40 2022] DELETE /identity/v3/users/910a9ffd40364d04b8789afb96496989 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:40.408685 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c766aaf-65cf-4b9d-a7b0-0a25eedea1d0 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:40.409128 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c766aaf-65cf-4b9d-a7b0-0a25eedea1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:40.409477 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c766aaf-65cf-4b9d-a7b0-0a25eedea1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:40.432323 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-2c766aaf-65cf-4b9d-a7b0-0a25eedea1d0 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:40.433226 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2c766aaf-65cf-4b9d-a7b0-0a25eedea1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:40.477880 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:40.507838 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1418/2866] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:40.517132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891e80b-b269-4ff4-b89b-67be9e012480 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:40.520073 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d891e80b-b269-4ff4-b89b-67be9e012480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:40.577233 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:40.593702 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891e80b-b269-4ff4-b89b-67be9e012480 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:40.598059 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf28749-96f8-4f06-a41a-ed6b439d089a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:40.598687 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf28749-96f8-4f06-a41a-ed6b439d089a service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:40.598968 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf28749-96f8-4f06-a41a-ed6b439d089a service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:40.599244 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf28749-96f8-4f06-a41a-ed6b439d089a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:40.609464 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf28749-96f8-4f06-a41a-ed6b439d089a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:40.611079 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf28749-96f8-4f06-a41a-ed6b439d089a service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:40.622098 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1449/2867] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:13:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:13:40.931788 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c766aaf-65cf-4b9d-a7b0-0a25eedea1d0 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:40.933262 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2c766aaf-65cf-4b9d-a7b0-0a25eedea1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:40.946596 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c766aaf-65cf-4b9d-a7b0-0a25eedea1d0 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:40.952055 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a569a-1cd4-4445-9eb3-57317f47d9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:40.952346 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a569a-1cd4-4445-9eb3-57317f47d9a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:40.952631 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a569a-1cd4-4445-9eb3-57317f47d9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:40.952906 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a569a-1cd4-4445-9eb3-57317f47d9a2 admin admin] PATH_INFO: `/v3/projects/81959ca275bd462fb319bc6a82f3cfdd` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:40.954146 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a569a-1cd4-4445-9eb3-57317f47d9a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81959ca275bd462fb319bc6a82f3cfdd)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:40.956111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a569a-1cd4-4445-9eb3-57317f47d9a2 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:41.009362 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-af0a569a-1cd4-4445-9eb3-57317f47d9a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81959ca275bd462fb319bc6a82f3cfdd', 'request_id': 'req-af0a569a-1cd4-4445-9eb3-57317f47d9a2'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:41.016295 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-af0a569a-1cd4-4445-9eb3-57317f47d9a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81959ca275bd462fb319bc6a82f3cfdd', 'request_id': 'req-af0a569a-1cd4-4445-9eb3-57317f47d9a2'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:41.023273 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-af0a569a-1cd4-4445-9eb3-57317f47d9a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81959ca275bd462fb319bc6a82f3cfdd', 'request_id': 'req-af0a569a-1cd4-4445-9eb3-57317f47d9a2'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:41.037393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-af0a569a-1cd4-4445-9eb3-57317f47d9a2 admin admin] The token cache is being invalidate because project 81959ca275bd462fb319bc6a82f3cfdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:13:41.037393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-af0a569a-1cd4-4445-9eb3-57317f47d9a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:41.037826 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-af0a569a-1cd4-4445-9eb3-57317f47d9a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:41.039394 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1419/2868] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:13:40 2022] DELETE /identity/v3/projects/81959ca275bd462fb319bc6a82f3cfdd => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:41.043811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf28749-96f8-4f06-a41a-ed6b439d089a service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:41.057986 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf28749-96f8-4f06-a41a-ed6b439d089a service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:41.136393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e78eaa-953b-426c-9e62-da9db1ea2f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:41.138405 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e78eaa-953b-426c-9e62-da9db1ea2f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:41.138405 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e78eaa-953b-426c-9e62-da9db1ea2f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:41.138405 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e78eaa-953b-426c-9e62-da9db1ea2f9a admin admin] PATH_INFO: `/v3/users/6e9c7d72d8d24e90ab8ba8902119782f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:41.139064 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e78eaa-953b-426c-9e62-da9db1ea2f9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e9c7d72d8d24e90ab8ba8902119782f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:41.141128 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e78eaa-953b-426c-9e62-da9db1ea2f9a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:41.192240 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f5e78eaa-953b-426c-9e62-da9db1ea2f9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e9c7d72d8d24e90ab8ba8902119782f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:41.193282 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f5e78eaa-953b-426c-9e62-da9db1ea2f9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e9c7d72d8d24e90ab8ba8902119782f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:41.201811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f5e78eaa-953b-426c-9e62-da9db1ea2f9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e9c7d72d8d24e90ab8ba8902119782f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:41.216124 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f5e78eaa-953b-426c-9e62-da9db1ea2f9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e9c7d72d8d24e90ab8ba8902119782f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:41.223899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f5e78eaa-953b-426c-9e62-da9db1ea2f9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e9c7d72d8d24e90ab8ba8902119782f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:41.230104 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1450/2869] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:13:41 2022] DELETE /identity/v3/users/6e9c7d72d8d24e90ab8ba8902119782f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:41.513056 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a569a-1cd4-4445-9eb3-57317f47d9a2 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:41.514635 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-af0a569a-1cd4-4445-9eb3-57317f47d9a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:41.527880 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a569a-1cd4-4445-9eb3-57317f47d9a2 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:41.593334 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b336d4-e9b7-4f48-ab2b-7ed2212eccc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:41.594158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b336d4-e9b7-4f48-ab2b-7ed2212eccc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:41.594549 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b336d4-e9b7-4f48-ab2b-7ed2212eccc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:41.594828 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b336d4-e9b7-4f48-ab2b-7ed2212eccc6 admin admin] PATH_INFO: `/v3/projects/e604fd3316e24d488cedcca1605b3393` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:41.596010 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b336d4-e9b7-4f48-ab2b-7ed2212eccc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e604fd3316e24d488cedcca1605b3393)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:41.598503 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b336d4-e9b7-4f48-ab2b-7ed2212eccc6 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:41.644630 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-34b336d4-e9b7-4f48-ab2b-7ed2212eccc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e604fd3316e24d488cedcca1605b3393', 'request_id': 'req-34b336d4-e9b7-4f48-ab2b-7ed2212eccc6'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:41.650615 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-34b336d4-e9b7-4f48-ab2b-7ed2212eccc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e604fd3316e24d488cedcca1605b3393', 'request_id': 'req-34b336d4-e9b7-4f48-ab2b-7ed2212eccc6'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:41.656219 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-34b336d4-e9b7-4f48-ab2b-7ed2212eccc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e604fd3316e24d488cedcca1605b3393', 'request_id': 'req-34b336d4-e9b7-4f48-ab2b-7ed2212eccc6'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:41.660418 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:41.670315 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-34b336d4-e9b7-4f48-ab2b-7ed2212eccc6 admin admin] The token cache is being invalidate because project e604fd3316e24d488cedcca1605b3393 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:13:41.670620 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-34b336d4-e9b7-4f48-ab2b-7ed2212eccc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:41.671340 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-34b336d4-e9b7-4f48-ab2b-7ed2212eccc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:41.673152 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1420/2870] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:13:41 2022] DELETE /identity/v3/projects/e604fd3316e24d488cedcca1605b3393 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:41.678695 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c377fce-cb54-4183-9ccc-4c5f3021135c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:41.678979 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c377fce-cb54-4183-9ccc-4c5f3021135c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:41.679252 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c377fce-cb54-4183-9ccc-4c5f3021135c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:41.698440 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8c377fce-cb54-4183-9ccc-4c5f3021135c None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:41.699630 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8c377fce-cb54-4183-9ccc-4c5f3021135c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:41.758734 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1451/2871] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:41.768531 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf52be-3e1a-42ec-90dd-87d13351ae88 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:41.768828 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf52be-3e1a-42ec-90dd-87d13351ae88 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:41.769254 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf52be-3e1a-42ec-90dd-87d13351ae88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:41.789495 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a9bf52be-3e1a-42ec-90dd-87d13351ae88 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:41.790740 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a9bf52be-3e1a-42ec-90dd-87d13351ae88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:41.798721 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1421/2872] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:41.803542 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c377fce-cb54-4183-9ccc-4c5f3021135c None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:41.814028 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c377fce-cb54-4183-9ccc-4c5f3021135c None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:41.819262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8750d-40d3-48ee-89a2-85348182f9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:41.819904 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8750d-40d3-48ee-89a2-85348182f9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:41.820186 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8750d-40d3-48ee-89a2-85348182f9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:41.820462 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8750d-40d3-48ee-89a2-85348182f9c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:41.821162 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8750d-40d3-48ee-89a2-85348182f9c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:13:41.821468 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8750d-40d3-48ee-89a2-85348182f9c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:41.823095 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8750d-40d3-48ee-89a2-85348182f9c8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:41.824587 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:41.837965 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1452/2873] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:13:41.842502 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf52be-3e1a-42ec-90dd-87d13351ae88 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:41.851923 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf52be-3e1a-42ec-90dd-87d13351ae88 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:41.856612 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddffa66-9346-4cee-a596-0748420ae849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:41.857222 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddffa66-9346-4cee-a596-0748420ae849 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:41.857504 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddffa66-9346-4cee-a596-0748420ae849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:41.857776 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddffa66-9346-4cee-a596-0748420ae849 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:41.858480 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddffa66-9346-4cee-a596-0748420ae849 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:41.860385 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddffa66-9346-4cee-a596-0748420ae849 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:41.890197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1422/2874] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:13:41 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:13:41.894180 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8750d-40d3-48ee-89a2-85348182f9c8 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:41.908171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8750d-40d3-48ee-89a2-85348182f9c8 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:41.965534 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898504c-4af1-4bd6-b907-57fa90182977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:41.966179 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898504c-4af1-4bd6-b907-57fa90182977 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:41.966457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898504c-4af1-4bd6-b907-57fa90182977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:41.966723 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898504c-4af1-4bd6-b907-57fa90182977 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:41.967381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898504c-4af1-4bd6-b907-57fa90182977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:41.969266 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898504c-4af1-4bd6-b907-57fa90182977 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:41.991055 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1453/2875] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:13:41 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:13:41.995469 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddffa66-9346-4cee-a596-0748420ae849 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:42.006251 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddffa66-9346-4cee-a596-0748420ae849 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:42.010928 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898510b-7853-4363-b78e-8a656dad9907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:42.011529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898510b-7853-4363-b78e-8a656dad9907 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:42.011808 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898510b-7853-4363-b78e-8a656dad9907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:42.012080 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898510b-7853-4363-b78e-8a656dad9907 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:42.012763 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898510b-7853-4363-b78e-8a656dad9907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:42.014401 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898510b-7853-4363-b78e-8a656dad9907 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:42.024322 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1423/2876] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:13:42.028475 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898504c-4af1-4bd6-b907-57fa90182977 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:42.038378 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898504c-4af1-4bd6-b907-57fa90182977 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:42.043377 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd535b-f927-4133-b21d-91caf20ff104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:42.044078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd535b-f927-4133-b21d-91caf20ff104 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:42.044361 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd535b-f927-4133-b21d-91caf20ff104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:42.044655 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd535b-f927-4133-b21d-91caf20ff104 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:42.045333 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd535b-f927-4133-b21d-91caf20ff104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:42.046967 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd535b-f927-4133-b21d-91caf20ff104 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:42.059092 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1454/2877] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:13:42.063552 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898510b-7853-4363-b78e-8a656dad9907 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:42.074754 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898510b-7853-4363-b78e-8a656dad9907 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:42.079660 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b7851-c832-429c-a246-edbffdcb4a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:42.080329 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b7851-c832-429c-a246-edbffdcb4a1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:42.080618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b7851-c832-429c-a246-edbffdcb4a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:42.080898 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b7851-c832-429c-a246-edbffdcb4a1d admin admin] PATH_INFO: `/v3/projects/192e31ba418941dcac164cecb186ccaf/users/2e22429e566f4b9db04089291e9539db/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:42.095983 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b7851-c832-429c-a246-edbffdcb4a1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=192e31ba418941dcac164cecb186ccaf, user_id=2e22429e566f4b9db04089291e9539db, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:42.098713 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b7851-c832-429c-a246-edbffdcb4a1d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:42.112253 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1424/2878] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:13:42 2022] PUT /identity/v3/projects/192e31ba418941dcac164cecb186ccaf/users/2e22429e566f4b9db04089291e9539db/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:13:42.123496 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb1a9c-4432-4485-9a68-e1ca27eabdbd None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:42.123788 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb1a9c-4432-4485-9a68-e1ca27eabdbd None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:42.124090 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb1a9c-4432-4485-9a68-e1ca27eabdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:42.152154 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-45fb1a9c-4432-4485-9a68-e1ca27eabdbd None None] MFA Rules not processed for user `2e22429e566f4b9db04089291e9539db`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:42.153371 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-45fb1a9c-4432-4485-9a68-e1ca27eabdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:42.217951 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1455/2879] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:42 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:42.223140 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b7851-c832-429c-a246-edbffdcb4a1d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:42.288031 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b7851-c832-429c-a246-edbffdcb4a1d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:42.292948 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d378b8-4b0b-40fa-95f0-eb65536ea1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:42.293577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d378b8-4b0b-40fa-95f0-eb65536ea1e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:42.293867 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d378b8-4b0b-40fa-95f0-eb65536ea1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:42.294182 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d378b8-4b0b-40fa-95f0-eb65536ea1e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:42.294860 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d378b8-4b0b-40fa-95f0-eb65536ea1e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:42.296774 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d378b8-4b0b-40fa-95f0-eb65536ea1e0 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:42.325308 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1425/2880] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:13:42 2022] POST /identity/v3/projects => generated 369 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:13:42.329432 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb1a9c-4432-4485-9a68-e1ca27eabdbd None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:42.340056 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb1a9c-4432-4485-9a68-e1ca27eabdbd None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:42.395806 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06d7b2-9854-40df-8c3c-818d8028a78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:42.396424 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06d7b2-9854-40df-8c3c-818d8028a78a admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:42.396707 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06d7b2-9854-40df-8c3c-818d8028a78a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:42.396978 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06d7b2-9854-40df-8c3c-818d8028a78a admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:42.397643 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06d7b2-9854-40df-8c3c-818d8028a78a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:42.399601 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06d7b2-9854-40df-8c3c-818d8028a78a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:42.408426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:42.422378 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1456/2881] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:13:42 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:13:42.426987 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d378b8-4b0b-40fa-95f0-eb65536ea1e0 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:42.437392 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d378b8-4b0b-40fa-95f0-eb65536ea1e0 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:42.442710 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e891f-7689-4ac6-967c-74ba0561ff22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:42.443332 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e891f-7689-4ac6-967c-74ba0561ff22 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:42.443611 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e891f-7689-4ac6-967c-74ba0561ff22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:42.444017 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e891f-7689-4ac6-967c-74ba0561ff22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:42.444716 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e891f-7689-4ac6-967c-74ba0561ff22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:42.446367 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e891f-7689-4ac6-967c-74ba0561ff22 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:42.458031 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1426/2882] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:13:42.462319 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06d7b2-9854-40df-8c3c-818d8028a78a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:42.473249 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06d7b2-9854-40df-8c3c-818d8028a78a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:42.477899 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fdd49-5cbb-4352-9c5d-2a2f1747ab2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:42.478581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fdd49-5cbb-4352-9c5d-2a2f1747ab2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:42.478863 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fdd49-5cbb-4352-9c5d-2a2f1747ab2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:42.479138 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fdd49-5cbb-4352-9c5d-2a2f1747ab2b admin admin] PATH_INFO: `/v3/projects/607be4df76744b27893b6a5c89313bf3/users/4d59e34062134127ad8594b028fe1e7d/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:42.494129 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87fdd49-5cbb-4352-9c5d-2a2f1747ab2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=607be4df76744b27893b6a5c89313bf3, user_id=4d59e34062134127ad8594b028fe1e7d, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:42.496778 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87fdd49-5cbb-4352-9c5d-2a2f1747ab2b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:42.510484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1457/2883] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:13:42 2022] PUT /identity/v3/projects/607be4df76744b27893b6a5c89313bf3/users/4d59e34062134127ad8594b028fe1e7d/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:42.514498 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e891f-7689-4ac6-967c-74ba0561ff22 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:42.525393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e891f-7689-4ac6-967c-74ba0561ff22 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:42.580588 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b2648-987f-4c66-ac9d-7b6b603711e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:42.581212 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b2648-987f-4c66-ac9d-7b6b603711e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:42.581492 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b2648-987f-4c66-ac9d-7b6b603711e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:42.582403 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b2648-987f-4c66-ac9d-7b6b603711e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:42.583094 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b2648-987f-4c66-ac9d-7b6b603711e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:42.584692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b2648-987f-4c66-ac9d-7b6b603711e3 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:42.594497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1427/2884] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:13:42.598657 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fdd49-5cbb-4352-9c5d-2a2f1747ab2b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:42.609026 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fdd49-5cbb-4352-9c5d-2a2f1747ab2b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:42.614356 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ce33e-7a4f-4087-ae46-f80aa1bf6bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:42.615019 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ce33e-7a4f-4087-ae46-f80aa1bf6bed admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:42.615302 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ce33e-7a4f-4087-ae46-f80aa1bf6bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:42.615581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ce33e-7a4f-4087-ae46-f80aa1bf6bed admin admin] PATH_INFO: `/v3/projects/607be4df76744b27893b6a5c89313bf3/users/4d59e34062134127ad8594b028fe1e7d/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:42.617817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ce33e-7a4f-4087-ae46-f80aa1bf6bed admin admin] RBAC: Authorizing `identity:create_grant(project_id=607be4df76744b27893b6a5c89313bf3, user_id=4d59e34062134127ad8594b028fe1e7d, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:42.620478 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ce33e-7a4f-4087-ae46-f80aa1bf6bed admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:42.632072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1458/2885] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:13:42 2022] PUT /identity/v3/projects/607be4df76744b27893b6a5c89313bf3/users/4d59e34062134127ad8594b028fe1e7d/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:42.643382 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de9944-53c7-4b46-b311-d863f6399408 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:42.643677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de9944-53c7-4b46-b311-d863f6399408 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:42.643958 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de9944-53c7-4b46-b311-d863f6399408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:42.651107 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:42.673513 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-56de9944-53c7-4b46-b311-d863f6399408 None None] MFA Rules not processed for user `4d59e34062134127ad8594b028fe1e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:42.674783 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-56de9944-53c7-4b46-b311-d863f6399408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:42.748088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1428/2886] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:42 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:42.761795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ce33e-7a4f-4087-ae46-f80aa1bf6bed admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:42.763712 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-714ce33e-7a4f-4087-ae46-f80aa1bf6bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:42.823310 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ce33e-7a4f-4087-ae46-f80aa1bf6bed admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:42.827293 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb5581-cb48-4b7c-86ff-1248fbbbf984 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:42.827906 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb5581-cb48-4b7c-86ff-1248fbbbf984 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:42.828191 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb5581-cb48-4b7c-86ff-1248fbbbf984 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:42.828467 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb5581-cb48-4b7c-86ff-1248fbbbf984 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:42.837484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb5581-cb48-4b7c-86ff-1248fbbbf984 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:42.838949 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb5581-cb48-4b7c-86ff-1248fbbbf984 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:42.848251 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1459/2887] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:13:42 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:13:42.895018 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de9944-53c7-4b46-b311-d863f6399408 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:42.896863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-56de9944-53c7-4b46-b311-d863f6399408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:42.912067 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de9944-53c7-4b46-b311-d863f6399408 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:42.916027 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3455d65-7f39-43cb-bc94-c201043be3ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:42.916656 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3455d65-7f39-43cb-bc94-c201043be3ba service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:42.916942 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3455d65-7f39-43cb-bc94-c201043be3ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:42.917229 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3455d65-7f39-43cb-bc94-c201043be3ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:42.975520 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3455d65-7f39-43cb-bc94-c201043be3ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:42.978011 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3455d65-7f39-43cb-bc94-c201043be3ba service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:42.987438 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1429/2888] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:13:42 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:13:46.322074 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb5581-cb48-4b7c-86ff-1248fbbbf984 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:46.383089 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb5581-cb48-4b7c-86ff-1248fbbbf984 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:46.387777 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ac53d-a4de-44f7-814a-32066281785a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:46.388407 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ac53d-a4de-44f7-814a-32066281785a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:46.388683 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ac53d-a4de-44f7-814a-32066281785a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:46.388948 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ac53d-a4de-44f7-814a-32066281785a admin admin] PATH_INFO: `/v3/users/2e22429e566f4b9db04089291e9539db` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:46.390135 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ac53d-a4de-44f7-814a-32066281785a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e22429e566f4b9db04089291e9539db)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:46.392030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ac53d-a4de-44f7-814a-32066281785a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:46.437731 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c03ac53d-a4de-44f7-814a-32066281785a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e22429e566f4b9db04089291e9539db'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:46.438408 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c03ac53d-a4de-44f7-814a-32066281785a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e22429e566f4b9db04089291e9539db'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:46.446181 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c03ac53d-a4de-44f7-814a-32066281785a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e22429e566f4b9db04089291e9539db'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:46.450867 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:INSERT=5,keystone:SELECT=87,keystone:DELETE=8 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:46.460729 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c03ac53d-a4de-44f7-814a-32066281785a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e22429e566f4b9db04089291e9539db'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:46.467811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c03ac53d-a4de-44f7-814a-32066281785a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e22429e566f4b9db04089291e9539db'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:46.473984 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1460/2889] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:13:46 2022] DELETE /identity/v3/users/2e22429e566f4b9db04089291e9539db => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:46.478448 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad198b-8dd8-443a-adc4-2937c9f79492 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:46.478937 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad198b-8dd8-443a-adc4-2937c9f79492 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:46.479026 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad198b-8dd8-443a-adc4-2937c9f79492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:46.498759 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-46ad198b-8dd8-443a-adc4-2937c9f79492 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:46.500677 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-46ad198b-8dd8-443a-adc4-2937c9f79492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:46.562309 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1430/2890] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:46.572814 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ac53d-a4de-44f7-814a-32066281785a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:46.574360 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c03ac53d-a4de-44f7-814a-32066281785a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:46.631638 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ac53d-a4de-44f7-814a-32066281785a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:46.635231 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d131583-f790-4a86-8217-0a1bdcdd1a66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:46.635845 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d131583-f790-4a86-8217-0a1bdcdd1a66 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:46.636123 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d131583-f790-4a86-8217-0a1bdcdd1a66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:46.636391 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d131583-f790-4a86-8217-0a1bdcdd1a66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:46.645548 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d131583-f790-4a86-8217-0a1bdcdd1a66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:46.647060 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d131583-f790-4a86-8217-0a1bdcdd1a66 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:46.656415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1461/2891] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:13:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:13:46.935161 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad198b-8dd8-443a-adc4-2937c9f79492 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:46.936754 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-46ad198b-8dd8-443a-adc4-2937c9f79492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:46.950909 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad198b-8dd8-443a-adc4-2937c9f79492 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:46.955763 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65382983-c0b6-4160-8975-990f07749299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:46.956432 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65382983-c0b6-4160-8975-990f07749299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:46.956863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65382983-c0b6-4160-8975-990f07749299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:46.957144 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65382983-c0b6-4160-8975-990f07749299 admin admin] PATH_INFO: `/v3/projects/192e31ba418941dcac164cecb186ccaf` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:46.958369 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65382983-c0b6-4160-8975-990f07749299 admin admin] RBAC: Authorizing `identity:delete_project(project_id=192e31ba418941dcac164cecb186ccaf)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:46.960322 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65382983-c0b6-4160-8975-990f07749299 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:46.984346 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:47.008839 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-65382983-c0b6-4160-8975-990f07749299 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '192e31ba418941dcac164cecb186ccaf', 'request_id': 'req-65382983-c0b6-4160-8975-990f07749299'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:47.015417 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-65382983-c0b6-4160-8975-990f07749299 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '192e31ba418941dcac164cecb186ccaf', 'request_id': 'req-65382983-c0b6-4160-8975-990f07749299'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:47.021003 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-65382983-c0b6-4160-8975-990f07749299 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '192e31ba418941dcac164cecb186ccaf', 'request_id': 'req-65382983-c0b6-4160-8975-990f07749299'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:47.032972 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-65382983-c0b6-4160-8975-990f07749299 admin admin] The token cache is being invalidate because project 192e31ba418941dcac164cecb186ccaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:13:47.033266 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-65382983-c0b6-4160-8975-990f07749299 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:47.033911 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-65382983-c0b6-4160-8975-990f07749299 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:47.035453 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1431/2892] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:13:46 2022] DELETE /identity/v3/projects/192e31ba418941dcac164cecb186ccaf => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:13:47.039971 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d131583-f790-4a86-8217-0a1bdcdd1a66 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:47.049555 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d131583-f790-4a86-8217-0a1bdcdd1a66 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:47.108741 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae026017-a549-4e58-92f5-c5fb161e8058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:47.109283 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae026017-a549-4e58-92f5-c5fb161e8058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:47.109560 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae026017-a549-4e58-92f5-c5fb161e8058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:47.109831 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae026017-a549-4e58-92f5-c5fb161e8058 admin admin] PATH_INFO: `/v3/users/4d59e34062134127ad8594b028fe1e7d` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:47.111006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae026017-a549-4e58-92f5-c5fb161e8058 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d59e34062134127ad8594b028fe1e7d)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:47.112921 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae026017-a549-4e58-92f5-c5fb161e8058 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:47.160163 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ae026017-a549-4e58-92f5-c5fb161e8058 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d59e34062134127ad8594b028fe1e7d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:47.160860 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ae026017-a549-4e58-92f5-c5fb161e8058 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d59e34062134127ad8594b028fe1e7d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:47.170517 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ae026017-a549-4e58-92f5-c5fb161e8058 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d59e34062134127ad8594b028fe1e7d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:47.184010 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ae026017-a549-4e58-92f5-c5fb161e8058 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d59e34062134127ad8594b028fe1e7d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:47.191134 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ae026017-a549-4e58-92f5-c5fb161e8058 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d59e34062134127ad8594b028fe1e7d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:47.192616 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:47.200608 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1462/2893] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:13:47 2022] DELETE /identity/v3/users/4d59e34062134127ad8594b028fe1e7d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:47.482876 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65382983-c0b6-4160-8975-990f07749299 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:47.484361 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-65382983-c0b6-4160-8975-990f07749299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:47.498145 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65382983-c0b6-4160-8975-990f07749299 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:47.563197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec65fb4-120c-4f55-91ab-b339305f1658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:47.564460 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec65fb4-120c-4f55-91ab-b339305f1658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:47.564765 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec65fb4-120c-4f55-91ab-b339305f1658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:47.565039 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec65fb4-120c-4f55-91ab-b339305f1658 admin admin] PATH_INFO: `/v3/projects/607be4df76744b27893b6a5c89313bf3` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:47.566282 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec65fb4-120c-4f55-91ab-b339305f1658 admin admin] RBAC: Authorizing `identity:delete_project(project_id=607be4df76744b27893b6a5c89313bf3)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:47.568218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec65fb4-120c-4f55-91ab-b339305f1658 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:47.617872 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0ec65fb4-120c-4f55-91ab-b339305f1658 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '607be4df76744b27893b6a5c89313bf3', 'request_id': 'req-0ec65fb4-120c-4f55-91ab-b339305f1658'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:47.624103 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0ec65fb4-120c-4f55-91ab-b339305f1658 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '607be4df76744b27893b6a5c89313bf3', 'request_id': 'req-0ec65fb4-120c-4f55-91ab-b339305f1658'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:47.630041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0ec65fb4-120c-4f55-91ab-b339305f1658 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '607be4df76744b27893b6a5c89313bf3', 'request_id': 'req-0ec65fb4-120c-4f55-91ab-b339305f1658'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:47.649111 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0ec65fb4-120c-4f55-91ab-b339305f1658 admin admin] The token cache is being invalidate because project 607be4df76744b27893b6a5c89313bf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:13:47.649407 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0ec65fb4-120c-4f55-91ab-b339305f1658 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:47.650079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0ec65fb4-120c-4f55-91ab-b339305f1658 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:47.651610 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1432/2894] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:13:47 2022] DELETE /identity/v3/projects/607be4df76744b27893b6a5c89313bf3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:47.656405 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022792c7-dbd6-43ff-b1c2-dfe9850098b5 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:47.656858 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022792c7-dbd6-43ff-b1c2-dfe9850098b5 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:47.656965 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022792c7-dbd6-43ff-b1c2-dfe9850098b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:47.677018 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-022792c7-dbd6-43ff-b1c2-dfe9850098b5 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:47.678817 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-022792c7-dbd6-43ff-b1c2-dfe9850098b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:47.736420 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1463/2895] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:47.745241 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700fb55-1585-4d09-a3b1-e8310c75f8c2 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:47.745560 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700fb55-1585-4d09-a3b1-e8310c75f8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:47.745842 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700fb55-1585-4d09-a3b1-e8310c75f8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:47.765791 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7700fb55-1585-4d09-a3b1-e8310c75f8c2 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:47.767025 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7700fb55-1585-4d09-a3b1-e8310c75f8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:47.775252 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1433/2896] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:47.779726 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022792c7-dbd6-43ff-b1c2-dfe9850098b5 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:47.790070 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022792c7-dbd6-43ff-b1c2-dfe9850098b5 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:47.794680 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc335e8-31aa-432b-b4b3-88aa23ad8e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:47.795296 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc335e8-31aa-432b-b4b3-88aa23ad8e4c admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:47.795574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc335e8-31aa-432b-b4b3-88aa23ad8e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:47.795854 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc335e8-31aa-432b-b4b3-88aa23ad8e4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:47.796545 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc335e8-31aa-432b-b4b3-88aa23ad8e4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:13:47.796851 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc335e8-31aa-432b-b4b3-88aa23ad8e4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:47.798476 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc335e8-31aa-432b-b4b3-88aa23ad8e4c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:47.810759 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1464/2897] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:13:47.815274 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700fb55-1585-4d09-a3b1-e8310c75f8c2 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:47.826307 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700fb55-1585-4d09-a3b1-e8310c75f8c2 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:47.831393 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6607544-6a4e-4bec-a740-9f7e84da7e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:47.831990 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6607544-6a4e-4bec-a740-9f7e84da7e3e admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:47.832275 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6607544-6a4e-4bec-a740-9f7e84da7e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:47.832555 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6607544-6a4e-4bec-a740-9f7e84da7e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:47.833863 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6607544-6a4e-4bec-a740-9f7e84da7e3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:47.836180 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6607544-6a4e-4bec-a740-9f7e84da7e3e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:47.865160 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1434/2898] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:13:47 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:13:47.870323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc335e8-31aa-432b-b4b3-88aa23ad8e4c admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:47.880375 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc335e8-31aa-432b-b4b3-88aa23ad8e4c admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:47.938874 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85de829-4bd3-4efe-aa33-8299461895b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:47.939498 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85de829-4bd3-4efe-aa33-8299461895b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:47.939784 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85de829-4bd3-4efe-aa33-8299461895b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:47.940054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85de829-4bd3-4efe-aa33-8299461895b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:47.940715 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85de829-4bd3-4efe-aa33-8299461895b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:47.942691 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85de829-4bd3-4efe-aa33-8299461895b3 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:47.964155 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1465/2899] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:13:47 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:13:47.968276 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6607544-6a4e-4bec-a740-9f7e84da7e3e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:47.979690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6607544-6a4e-4bec-a740-9f7e84da7e3e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:47.984483 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e72cf1-a046-489b-8d96-479ab9a2ff55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:47.985090 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e72cf1-a046-489b-8d96-479ab9a2ff55 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:47.985373 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e72cf1-a046-489b-8d96-479ab9a2ff55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:47.985647 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e72cf1-a046-489b-8d96-479ab9a2ff55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:47.986380 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e72cf1-a046-489b-8d96-479ab9a2ff55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:47.988009 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e72cf1-a046-489b-8d96-479ab9a2ff55 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:47.993570 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:47.999270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1435/2900] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:13:48.003072 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85de829-4bd3-4efe-aa33-8299461895b3 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:48.012913 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85de829-4bd3-4efe-aa33-8299461895b3 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:48.017652 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675513b-039d-4d17-bd76-9693de19572b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:48.018309 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675513b-039d-4d17-bd76-9693de19572b admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:48.018590 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675513b-039d-4d17-bd76-9693de19572b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:48.018859 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675513b-039d-4d17-bd76-9693de19572b admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:48.019544 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675513b-039d-4d17-bd76-9693de19572b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:48.021670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675513b-039d-4d17-bd76-9693de19572b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:48.032374 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1466/2901] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:13:48.037394 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e72cf1-a046-489b-8d96-479ab9a2ff55 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:48.047326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e72cf1-a046-489b-8d96-479ab9a2ff55 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:48.051937 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e3653-89af-443a-b479-b008d06a92fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:48.052613 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e3653-89af-443a-b479-b008d06a92fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:48.052896 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e3653-89af-443a-b479-b008d06a92fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:48.053174 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e3653-89af-443a-b479-b008d06a92fb admin admin] PATH_INFO: `/v3/projects/6df23b4b7f284e6b80b0315a0f8f348c/users/c1cbf1e306594edaa14bd06243514788/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:48.068240 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e3653-89af-443a-b479-b008d06a92fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6df23b4b7f284e6b80b0315a0f8f348c, user_id=c1cbf1e306594edaa14bd06243514788, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:48.070929 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e3653-89af-443a-b479-b008d06a92fb admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:48.091229 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1436/2902] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:13:48 2022] PUT /identity/v3/projects/6df23b4b7f284e6b80b0315a0f8f348c/users/c1cbf1e306594edaa14bd06243514788/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:48.102690 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f6fd8-3936-4edd-a2cb-ac1e9164052b None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:48.103061 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f6fd8-3936-4edd-a2cb-ac1e9164052b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:48.103347 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f6fd8-3936-4edd-a2cb-ac1e9164052b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:48.132763 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-da6f6fd8-3936-4edd-a2cb-ac1e9164052b None None] MFA Rules not processed for user `c1cbf1e306594edaa14bd06243514788`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:48.133937 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-da6f6fd8-3936-4edd-a2cb-ac1e9164052b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:48.187593 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:48.199337 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1467/2903] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:48.203899 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e3653-89af-443a-b479-b008d06a92fb admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:48.264719 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e3653-89af-443a-b479-b008d06a92fb admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:48.269451 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5de89-a251-4a59-83f7-08b2e8db526c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:48.270099 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5de89-a251-4a59-83f7-08b2e8db526c admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:48.270386 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5de89-a251-4a59-83f7-08b2e8db526c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:48.270662 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5de89-a251-4a59-83f7-08b2e8db526c admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:48.271879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5de89-a251-4a59-83f7-08b2e8db526c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:48.273848 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5de89-a251-4a59-83f7-08b2e8db526c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:48.301211 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1437/2904] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:13:48 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:13:48.305285 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f6fd8-3936-4edd-a2cb-ac1e9164052b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:48.316229 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f6fd8-3936-4edd-a2cb-ac1e9164052b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:48.374671 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3602d-02b6-4d97-aecc-9c9a0a8816ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:48.375287 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3602d-02b6-4d97-aecc-9c9a0a8816ec admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:48.375568 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3602d-02b6-4d97-aecc-9c9a0a8816ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:48.375836 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3602d-02b6-4d97-aecc-9c9a0a8816ec admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:48.376499 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3602d-02b6-4d97-aecc-9c9a0a8816ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:48.378431 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3602d-02b6-4d97-aecc-9c9a0a8816ec admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:48.400201 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1468/2905] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:13:48 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:13:48.404190 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5de89-a251-4a59-83f7-08b2e8db526c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:48.414652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5de89-a251-4a59-83f7-08b2e8db526c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:48.419312 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3af233-6450-4dce-9384-f7958d9dfa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:48.419919 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3af233-6450-4dce-9384-f7958d9dfa9a admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:48.420342 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3af233-6450-4dce-9384-f7958d9dfa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:48.420621 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3af233-6450-4dce-9384-f7958d9dfa9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:48.421339 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3af233-6450-4dce-9384-f7958d9dfa9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:48.422985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3af233-6450-4dce-9384-f7958d9dfa9a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:48.433579 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1438/2906] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:13:48.437694 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3602d-02b6-4d97-aecc-9c9a0a8816ec admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:48.448528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3602d-02b6-4d97-aecc-9c9a0a8816ec admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:48.453389 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787d952-17e7-4200-9ecc-a17bb5c2d7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:48.454100 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787d952-17e7-4200-9ecc-a17bb5c2d7d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:48.454393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787d952-17e7-4200-9ecc-a17bb5c2d7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:48.454670 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787d952-17e7-4200-9ecc-a17bb5c2d7d3 admin admin] PATH_INFO: `/v3/projects/b4ce54889abd4662a2068111345a0309/users/417c936e198b401ab1194653427d0e95/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:48.470491 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787d952-17e7-4200-9ecc-a17bb5c2d7d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4ce54889abd4662a2068111345a0309, user_id=417c936e198b401ab1194653427d0e95, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:48.473116 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787d952-17e7-4200-9ecc-a17bb5c2d7d3 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:48.485585 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1469/2907] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:13:48 2022] PUT /identity/v3/projects/b4ce54889abd4662a2068111345a0309/users/417c936e198b401ab1194653427d0e95/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:48.489500 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3af233-6450-4dce-9384-f7958d9dfa9a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:48.499945 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3af233-6450-4dce-9384-f7958d9dfa9a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:48.560061 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5266ecb-f882-4aea-9fb5-9bdfa4d2a214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:48.560693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5266ecb-f882-4aea-9fb5-9bdfa4d2a214 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:48.560977 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5266ecb-f882-4aea-9fb5-9bdfa4d2a214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:48.561255 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5266ecb-f882-4aea-9fb5-9bdfa4d2a214 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:48.562022 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5266ecb-f882-4aea-9fb5-9bdfa4d2a214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:48.563622 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5266ecb-f882-4aea-9fb5-9bdfa4d2a214 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:48.574292 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1439/2908] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:13:48.578473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787d952-17e7-4200-9ecc-a17bb5c2d7d3 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:48.588677 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787d952-17e7-4200-9ecc-a17bb5c2d7d3 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:48.593214 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101cbb3b-5143-4d6a-8d68-13791c7c87bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:48.593858 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101cbb3b-5143-4d6a-8d68-13791c7c87bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:48.594256 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101cbb3b-5143-4d6a-8d68-13791c7c87bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:48.594445 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101cbb3b-5143-4d6a-8d68-13791c7c87bc admin admin] PATH_INFO: `/v3/projects/b4ce54889abd4662a2068111345a0309/users/417c936e198b401ab1194653427d0e95/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:48.596365 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101cbb3b-5143-4d6a-8d68-13791c7c87bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4ce54889abd4662a2068111345a0309, user_id=417c936e198b401ab1194653427d0e95, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:48.599006 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101cbb3b-5143-4d6a-8d68-13791c7c87bc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:48.608902 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1470/2909] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:13:48 2022] PUT /identity/v3/projects/b4ce54889abd4662a2068111345a0309/users/417c936e198b401ab1194653427d0e95/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:48.620892 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab3636-ee8b-4fea-99fa-09e6072361b3 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:48.621333 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab3636-ee8b-4fea-99fa-09e6072361b3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:48.621462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab3636-ee8b-4fea-99fa-09e6072361b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:48.649684 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e4ab3636-ee8b-4fea-99fa-09e6072361b3 None None] MFA Rules not processed for user `417c936e198b401ab1194653427d0e95`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:48.650991 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e4ab3636-ee8b-4fea-99fa-09e6072361b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:48.683938 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:48.719529 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1440/2910] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:13:48.731558 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101cbb3b-5143-4d6a-8d68-13791c7c87bc admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:48.733433 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-101cbb3b-5143-4d6a-8d68-13791c7c87bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:48.793130 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101cbb3b-5143-4d6a-8d68-13791c7c87bc admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:48.796897 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acae560-c181-41d1-a44c-b47ea4b66a36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:48.797519 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acae560-c181-41d1-a44c-b47ea4b66a36 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:48.797794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acae560-c181-41d1-a44c-b47ea4b66a36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:48.798099 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acae560-c181-41d1-a44c-b47ea4b66a36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:48.807794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acae560-c181-41d1-a44c-b47ea4b66a36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:48.809219 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acae560-c181-41d1-a44c-b47ea4b66a36 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:48.819310 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1471/2911] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:13:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:13:49.150890 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab3636-ee8b-4fea-99fa-09e6072361b3 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:49.160136 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab3636-ee8b-4fea-99fa-09e6072361b3 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:49.165121 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acd476-0fe1-4109-b2f9-668866df05c0 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417c936e198b401ab1194653427d0e95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4ce54889abd4662a2068111345a0309', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:49.165733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acd476-0fe1-4109-b2f9-668866df05c0 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:49.166063 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acd476-0fe1-4109-b2f9-668866df05c0 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:49.166348 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acd476-0fe1-4109-b2f9-668866df05c0 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:49.167044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acd476-0fe1-4109-b2f9-668866df05c0 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:13:49.167352 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acd476-0fe1-4109-b2f9-668866df05c0 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:49.168962 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acd476-0fe1-4109-b2f9-668866df05c0 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:49.181319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1441/2912] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:13:49.185626 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acae560-c181-41d1-a44c-b47ea4b66a36 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:49.194698 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acae560-c181-41d1-a44c-b47ea4b66a36 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:49.199230 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d48e1b-f3a2-4b75-99ef-9e30843eb067 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417c936e198b401ab1194653427d0e95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4ce54889abd4662a2068111345a0309', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:49.201605 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d48e1b-f3a2-4b75-99ef-9e30843eb067 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:49.201909 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d48e1b-f3a2-4b75-99ef-9e30843eb067 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:49.202197 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d48e1b-f3a2-4b75-99ef-9e30843eb067 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:49.202868 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d48e1b-f3a2-4b75-99ef-9e30843eb067 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:49.204752 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d48e1b-f3a2-4b75-99ef-9e30843eb067 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:49.232530 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1472/2913] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:13:49 2022] POST /identity/v3/projects => generated 354 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:13:49.237115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acd476-0fe1-4109-b2f9-668866df05c0 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:49.248396 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acd476-0fe1-4109-b2f9-668866df05c0 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:49.315021 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b69f6-7f17-4d57-84af-f02cded0abf2 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417c936e198b401ab1194653427d0e95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4ce54889abd4662a2068111345a0309', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:49.315653 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b69f6-7f17-4d57-84af-f02cded0abf2 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:49.315943 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b69f6-7f17-4d57-84af-f02cded0abf2 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:49.316225 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b69f6-7f17-4d57-84af-f02cded0abf2 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:49.316923 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b69f6-7f17-4d57-84af-f02cded0abf2 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:13:49.317231 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b69f6-7f17-4d57-84af-f02cded0abf2 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:49.318879 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b69f6-7f17-4d57-84af-f02cded0abf2 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:49.331122 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1442/2914] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:13:49.371682 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d48e1b-f3a2-4b75-99ef-9e30843eb067 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:49.382172 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d48e1b-f3a2-4b75-99ef-9e30843eb067 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:49.386768 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404eb36f-8e68-48e0-b473-32ddd7b52f28 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417c936e198b401ab1194653427d0e95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4ce54889abd4662a2068111345a0309', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:49.387401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404eb36f-8e68-48e0-b473-32ddd7b52f28 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:49.387687 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404eb36f-8e68-48e0-b473-32ddd7b52f28 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:49.388101 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404eb36f-8e68-48e0-b473-32ddd7b52f28 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] PATH_INFO: `/v3/projects/201e4e2ae60d44bd91eb6c9ce32f433f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:49.389281 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404eb36f-8e68-48e0-b473-32ddd7b52f28 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] RBAC: Authorizing `identity:delete_project(project_id=201e4e2ae60d44bd91eb6c9ce32f433f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:49.391215 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404eb36f-8e68-48e0-b473-32ddd7b52f28 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:49.415332 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:49.439313 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-404eb36f-8e68-48e0-b473-32ddd7b52f28 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '201e4e2ae60d44bd91eb6c9ce32f433f', 'request_id': 'req-404eb36f-8e68-48e0-b473-32ddd7b52f28'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:49.444776 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-404eb36f-8e68-48e0-b473-32ddd7b52f28 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '201e4e2ae60d44bd91eb6c9ce32f433f', 'request_id': 'req-404eb36f-8e68-48e0-b473-32ddd7b52f28'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:49.451462 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-404eb36f-8e68-48e0-b473-32ddd7b52f28 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '201e4e2ae60d44bd91eb6c9ce32f433f', 'request_id': 'req-404eb36f-8e68-48e0-b473-32ddd7b52f28'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:49.464200 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-404eb36f-8e68-48e0-b473-32ddd7b52f28 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] The token cache is being invalidate because project 201e4e2ae60d44bd91eb6c9ce32f433f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:13:49.464491 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-404eb36f-8e68-48e0-b473-32ddd7b52f28 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:49.465135 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-404eb36f-8e68-48e0-b473-32ddd7b52f28 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:49.466873 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1473/2915] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:13:49 2022] DELETE /identity/v3/projects/201e4e2ae60d44bd91eb6c9ce32f433f => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:49.666578 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b69f6-7f17-4d57-84af-f02cded0abf2 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:49.668238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-cb7b69f6-7f17-4d57-84af-f02cded0abf2 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:49.731654 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b69f6-7f17-4d57-84af-f02cded0abf2 tempest-VolumeQuotaClassesTest-382830338 tempest-VolumeQuotaClassesTest-382830338-project-admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:49.787164 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320377c-9ce3-438e-bbd4-3ba4a9e8cc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:49.787812 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320377c-9ce3-438e-bbd4-3ba4a9e8cc79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:49.788096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320377c-9ce3-438e-bbd4-3ba4a9e8cc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:49.788370 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320377c-9ce3-438e-bbd4-3ba4a9e8cc79 admin admin] PATH_INFO: `/v3/users/c1cbf1e306594edaa14bd06243514788` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:49.789550 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320377c-9ce3-438e-bbd4-3ba4a9e8cc79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1cbf1e306594edaa14bd06243514788)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:49.791510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320377c-9ce3-438e-bbd4-3ba4a9e8cc79 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:49.792996 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:49.840256 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3320377c-9ce3-438e-bbd4-3ba4a9e8cc79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1cbf1e306594edaa14bd06243514788'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:49.840912 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3320377c-9ce3-438e-bbd4-3ba4a9e8cc79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1cbf1e306594edaa14bd06243514788'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:49.848595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3320377c-9ce3-438e-bbd4-3ba4a9e8cc79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1cbf1e306594edaa14bd06243514788'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:49.860785 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3320377c-9ce3-438e-bbd4-3ba4a9e8cc79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1cbf1e306594edaa14bd06243514788'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:49.861449 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3320377c-9ce3-438e-bbd4-3ba4a9e8cc79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1cbf1e306594edaa14bd06243514788'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:49.873827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1443/2916] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:13:49 2022] DELETE /identity/v3/users/c1cbf1e306594edaa14bd06243514788 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:49.878339 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29e268-dbb5-45cf-bcb3-b63ffaa7558d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:49.878811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29e268-dbb5-45cf-bcb3-b63ffaa7558d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:49.878920 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29e268-dbb5-45cf-bcb3-b63ffaa7558d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:49.898458 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-ca29e268-dbb5-45cf-bcb3-b63ffaa7558d None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:49.899673 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ca29e268-dbb5-45cf-bcb3-b63ffaa7558d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:49.957983 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1474/2917] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:49.968426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320377c-9ce3-438e-bbd4-3ba4a9e8cc79 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:49.969973 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3320377c-9ce3-438e-bbd4-3ba4a9e8cc79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:50.027493 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320377c-9ce3-438e-bbd4-3ba4a9e8cc79 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:50.032010 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d77ba-81df-40d2-9a67-b6e6cc335b18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:50.032758 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d77ba-81df-40d2-9a67-b6e6cc335b18 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:50.033120 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d77ba-81df-40d2-9a67-b6e6cc335b18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:50.033462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d77ba-81df-40d2-9a67-b6e6cc335b18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:50.042732 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d77ba-81df-40d2-9a67-b6e6cc335b18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:50.044174 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d77ba-81df-40d2-9a67-b6e6cc335b18 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:50.055115 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1444/2918] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:13:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:13:50.324740 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca29e268-dbb5-45cf-bcb3-b63ffaa7558d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:50.326861 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ca29e268-dbb5-45cf-bcb3-b63ffaa7558d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:50.347928 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca29e268-dbb5-45cf-bcb3-b63ffaa7558d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:50.352881 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71cefe-c300-48f8-876f-7fc17240c906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:50.353426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71cefe-c300-48f8-876f-7fc17240c906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:50.353703 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71cefe-c300-48f8-876f-7fc17240c906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:50.354021 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71cefe-c300-48f8-876f-7fc17240c906 admin admin] PATH_INFO: `/v3/projects/6df23b4b7f284e6b80b0315a0f8f348c` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:50.355311 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71cefe-c300-48f8-876f-7fc17240c906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6df23b4b7f284e6b80b0315a0f8f348c)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:50.357199 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71cefe-c300-48f8-876f-7fc17240c906 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:50.405509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1a71cefe-c300-48f8-876f-7fc17240c906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6df23b4b7f284e6b80b0315a0f8f348c', 'request_id': 'req-1a71cefe-c300-48f8-876f-7fc17240c906'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:50.410894 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1a71cefe-c300-48f8-876f-7fc17240c906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6df23b4b7f284e6b80b0315a0f8f348c', 'request_id': 'req-1a71cefe-c300-48f8-876f-7fc17240c906'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:50.417481 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1a71cefe-c300-48f8-876f-7fc17240c906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6df23b4b7f284e6b80b0315a0f8f348c', 'request_id': 'req-1a71cefe-c300-48f8-876f-7fc17240c906'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:50.430841 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1a71cefe-c300-48f8-876f-7fc17240c906 admin admin] The token cache is being invalidate because project 6df23b4b7f284e6b80b0315a0f8f348c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:13:50.431123 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1a71cefe-c300-48f8-876f-7fc17240c906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:50.431755 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-1a71cefe-c300-48f8-876f-7fc17240c906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:50.433298 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1475/2919] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:13:50 2022] DELETE /identity/v3/projects/6df23b4b7f284e6b80b0315a0f8f348c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:50.437349 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d77ba-81df-40d2-9a67-b6e6cc335b18 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:50.447085 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d77ba-81df-40d2-9a67-b6e6cc335b18 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:50.504255 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b96c1-7cb9-4eb8-8a6e-c67048bc905b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:50.504822 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b96c1-7cb9-4eb8-8a6e-c67048bc905b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:50.505109 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b96c1-7cb9-4eb8-8a6e-c67048bc905b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:50.505415 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b96c1-7cb9-4eb8-8a6e-c67048bc905b admin admin] PATH_INFO: `/v3/users/417c936e198b401ab1194653427d0e95` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:50.506610 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b96c1-7cb9-4eb8-8a6e-c67048bc905b admin admin] RBAC: Authorizing `identity:delete_user(user_id=417c936e198b401ab1194653427d0e95)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:50.508543 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b96c1-7cb9-4eb8-8a6e-c67048bc905b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:50.530583 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:50.556369 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ae6b96c1-7cb9-4eb8-8a6e-c67048bc905b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '417c936e198b401ab1194653427d0e95'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:50.557024 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ae6b96c1-7cb9-4eb8-8a6e-c67048bc905b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '417c936e198b401ab1194653427d0e95'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:50.564388 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ae6b96c1-7cb9-4eb8-8a6e-c67048bc905b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '417c936e198b401ab1194653427d0e95'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:50.576371 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ae6b96c1-7cb9-4eb8-8a6e-c67048bc905b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '417c936e198b401ab1194653427d0e95'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:50.577029 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ae6b96c1-7cb9-4eb8-8a6e-c67048bc905b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '417c936e198b401ab1194653427d0e95'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:50.589318 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1445/2920] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:13:50 2022] DELETE /identity/v3/users/417c936e198b401ab1194653427d0e95 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:50.873996 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71cefe-c300-48f8-876f-7fc17240c906 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:50.875629 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1a71cefe-c300-48f8-876f-7fc17240c906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:50.891262 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71cefe-c300-48f8-876f-7fc17240c906 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:50.929637 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:50.956843 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df51a3d-cce3-4c54-85e3-871bb7f7f435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:50.957489 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df51a3d-cce3-4c54-85e3-871bb7f7f435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:50.957769 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df51a3d-cce3-4c54-85e3-871bb7f7f435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:50.958076 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df51a3d-cce3-4c54-85e3-871bb7f7f435 admin admin] PATH_INFO: `/v3/projects/b4ce54889abd4662a2068111345a0309` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:50.960171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df51a3d-cce3-4c54-85e3-871bb7f7f435 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4ce54889abd4662a2068111345a0309)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:50.962106 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df51a3d-cce3-4c54-85e3-871bb7f7f435 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:51.010984 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-4df51a3d-cce3-4c54-85e3-871bb7f7f435 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4ce54889abd4662a2068111345a0309', 'request_id': 'req-4df51a3d-cce3-4c54-85e3-871bb7f7f435'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:51.016205 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-4df51a3d-cce3-4c54-85e3-871bb7f7f435 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4ce54889abd4662a2068111345a0309', 'request_id': 'req-4df51a3d-cce3-4c54-85e3-871bb7f7f435'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:51.023511 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-4df51a3d-cce3-4c54-85e3-871bb7f7f435 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4ce54889abd4662a2068111345a0309', 'request_id': 'req-4df51a3d-cce3-4c54-85e3-871bb7f7f435'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:51.038058 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-4df51a3d-cce3-4c54-85e3-871bb7f7f435 admin admin] The token cache is being invalidate because project b4ce54889abd4662a2068111345a0309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:13:51.038343 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-4df51a3d-cce3-4c54-85e3-871bb7f7f435 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:51.038956 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-4df51a3d-cce3-4c54-85e3-871bb7f7f435 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:51.040449 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1476/2921] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:13:50 2022] DELETE /identity/v3/projects/b4ce54889abd4662a2068111345a0309 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:51.046321 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eba58-7f87-4540-827a-d62ef33b2530 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:51.046781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eba58-7f87-4540-827a-d62ef33b2530 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:51.046875 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eba58-7f87-4540-827a-d62ef33b2530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:51.067389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7f8eba58-7f87-4540-827a-d62ef33b2530 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:51.068613 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7f8eba58-7f87-4540-827a-d62ef33b2530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:51.136725 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1446/2922] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:51.143739 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc418595-22ec-4203-b325-8c8842ac6470 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:51.144030 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc418595-22ec-4203-b325-8c8842ac6470 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:51.144300 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc418595-22ec-4203-b325-8c8842ac6470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:51.165017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-bc418595-22ec-4203-b325-8c8842ac6470 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:51.166251 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-bc418595-22ec-4203-b325-8c8842ac6470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:51.174132 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1477/2923] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:51.178532 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8eba58-7f87-4540-827a-d62ef33b2530 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:51.188297 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8eba58-7f87-4540-827a-d62ef33b2530 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:51.193401 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914ce06-edbc-4176-b6f6-289a4ab46e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:51.194044 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914ce06-edbc-4176-b6f6-289a4ab46e9e admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:51.194330 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914ce06-edbc-4176-b6f6-289a4ab46e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:51.194605 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914ce06-edbc-4176-b6f6-289a4ab46e9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:51.195304 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914ce06-edbc-4176-b6f6-289a4ab46e9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:13:51.195612 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914ce06-edbc-4176-b6f6-289a4ab46e9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:51.197200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914ce06-edbc-4176-b6f6-289a4ab46e9e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:51.209278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1447/2924] 10.222.0.83 () {58 vars in 1153 bytes} [Mon Aug 1 16:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:13:51.213645 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc418595-22ec-4203-b325-8c8842ac6470 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:51.222819 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc418595-22ec-4203-b325-8c8842ac6470 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:51.227417 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ff50a-1fc4-4f3b-9f08-527d09de4edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:51.228019 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ff50a-1fc4-4f3b-9f08-527d09de4edc admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:51.228299 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ff50a-1fc4-4f3b-9f08-527d09de4edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:51.228569 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ff50a-1fc4-4f3b-9f08-527d09de4edc admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:51.229223 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ff50a-1fc4-4f3b-9f08-527d09de4edc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:51.231126 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ff50a-1fc4-4f3b-9f08-527d09de4edc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:51.259721 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1478/2925] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:13:51 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:13:51.264210 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914ce06-edbc-4176-b6f6-289a4ab46e9e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:51.274930 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914ce06-edbc-4176-b6f6-289a4ab46e9e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:51.336681 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62b11c-a447-4a39-961d-bcfc59a7cdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:51.337319 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62b11c-a447-4a39-961d-bcfc59a7cdf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:51.337604 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62b11c-a447-4a39-961d-bcfc59a7cdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:51.337881 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62b11c-a447-4a39-961d-bcfc59a7cdf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:51.338575 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62b11c-a447-4a39-961d-bcfc59a7cdf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:51.340495 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62b11c-a447-4a39-961d-bcfc59a7cdf7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:51.361105 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1448/2926] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:13:51 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:13:51.365336 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ff50a-1fc4-4f3b-9f08-527d09de4edc admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:51.375867 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ff50a-1fc4-4f3b-9f08-527d09de4edc admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:51.381012 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1132e2-0b51-4022-8dfa-5e75012a7107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:51.381621 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1132e2-0b51-4022-8dfa-5e75012a7107 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:51.381915 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1132e2-0b51-4022-8dfa-5e75012a7107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:51.382197 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1132e2-0b51-4022-8dfa-5e75012a7107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:51.382886 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1132e2-0b51-4022-8dfa-5e75012a7107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:51.384466 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1132e2-0b51-4022-8dfa-5e75012a7107 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:51.394425 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1479/2927] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:13:51.399158 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62b11c-a447-4a39-961d-bcfc59a7cdf7 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:51.409462 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62b11c-a447-4a39-961d-bcfc59a7cdf7 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:51.414526 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e567ab-d1d3-432d-9249-a8e6b76b5fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:51.415140 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e567ab-d1d3-432d-9249-a8e6b76b5fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:51.415426 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e567ab-d1d3-432d-9249-a8e6b76b5fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:51.415733 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e567ab-d1d3-432d-9249-a8e6b76b5fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:51.416445 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e567ab-d1d3-432d-9249-a8e6b76b5fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:51.418096 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e567ab-d1d3-432d-9249-a8e6b76b5fe8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:51.428687 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1449/2928] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:13:51.434617 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1132e2-0b51-4022-8dfa-5e75012a7107 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:51.445763 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1132e2-0b51-4022-8dfa-5e75012a7107 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:51.450452 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03893ed-259b-4777-936c-9c4d5aaeedf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:51.451145 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03893ed-259b-4777-936c-9c4d5aaeedf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:51.451426 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03893ed-259b-4777-936c-9c4d5aaeedf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:51.451702 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03893ed-259b-4777-936c-9c4d5aaeedf7 admin admin] PATH_INFO: `/v3/projects/f6b67ce7dcce4825b2b4747425f19441/users/9a20d6a034c2449daaf05da97b2feb5e/roles/9cda4e2827dc4c2e87477bef8f319f92` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:51.466791 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03893ed-259b-4777-936c-9c4d5aaeedf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6b67ce7dcce4825b2b4747425f19441, user_id=9a20d6a034c2449daaf05da97b2feb5e, role_id=9cda4e2827dc4c2e87477bef8f319f92)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:51.469415 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03893ed-259b-4777-936c-9c4d5aaeedf7 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:51.482351 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1480/2929] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:13:51 2022] PUT /identity/v3/projects/f6b67ce7dcce4825b2b4747425f19441/users/9a20d6a034c2449daaf05da97b2feb5e/roles/9cda4e2827dc4c2e87477bef8f319f92 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:51.494405 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae6137-42b7-4a08-b74d-8a79276381b0 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:51.494692 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae6137-42b7-4a08-b74d-8a79276381b0 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:51.494962 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae6137-42b7-4a08-b74d-8a79276381b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:51.508792 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:51.524823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-2aae6137-42b7-4a08-b74d-8a79276381b0 None None] MFA Rules not processed for user `9a20d6a034c2449daaf05da97b2feb5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:51.526020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2aae6137-42b7-4a08-b74d-8a79276381b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:51.590978 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1450/2930] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:51.595795 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03893ed-259b-4777-936c-9c4d5aaeedf7 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:51.656974 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03893ed-259b-4777-936c-9c4d5aaeedf7 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:51.661545 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cbb38-add2-4403-9667-1858d7c89b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:51.662209 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cbb38-add2-4403-9667-1858d7c89b86 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:51.662484 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cbb38-add2-4403-9667-1858d7c89b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:51.662753 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cbb38-add2-4403-9667-1858d7c89b86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:51.663404 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cbb38-add2-4403-9667-1858d7c89b86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:51.665292 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cbb38-add2-4403-9667-1858d7c89b86 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:51.669802 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:51.694901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1481/2931] 10.222.0.83 () {60 vars in 1153 bytes} [Mon Aug 1 16:13:51 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:13:51.698920 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae6137-42b7-4a08-b74d-8a79276381b0 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:51.709690 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae6137-42b7-4a08-b74d-8a79276381b0 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:51.765769 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded4f75-f94c-47ce-bb2c-a4270c73385d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:51.766428 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded4f75-f94c-47ce-bb2c-a4270c73385d admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:51.766715 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded4f75-f94c-47ce-bb2c-a4270c73385d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:51.766995 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded4f75-f94c-47ce-bb2c-a4270c73385d admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:51.767666 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded4f75-f94c-47ce-bb2c-a4270c73385d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:51.769602 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded4f75-f94c-47ce-bb2c-a4270c73385d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:51.791418 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1451/2932] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Aug 1 16:13:51 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:13:51.795648 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cbb38-add2-4403-9667-1858d7c89b86 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:51.806448 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cbb38-add2-4403-9667-1858d7c89b86 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:51.811066 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5a6c5-f056-471b-b929-5d14c39e3b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:51.811676 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5a6c5-f056-471b-b929-5d14c39e3b8e admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:51.811954 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5a6c5-f056-471b-b929-5d14c39e3b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:51.812226 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5a6c5-f056-471b-b929-5d14c39e3b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:51.812908 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f5a6c5-f056-471b-b929-5d14c39e3b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:51.814528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f5a6c5-f056-471b-b929-5d14c39e3b8e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:51.824984 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1482/2933] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:13:51.829082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded4f75-f94c-47ce-bb2c-a4270c73385d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:51.839354 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded4f75-f94c-47ce-bb2c-a4270c73385d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:51.843886 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834c715-7ff0-4423-a5ec-6ed6864ecf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:51.844535 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834c715-7ff0-4423-a5ec-6ed6864ecf81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:51.844816 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834c715-7ff0-4423-a5ec-6ed6864ecf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:51.845093 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834c715-7ff0-4423-a5ec-6ed6864ecf81 admin admin] PATH_INFO: `/v3/projects/b7990cbe5c1641369d4de0d5c2d4c66b/users/0a9e7a884fcf4d21919fff85e78e4ad8/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:51.859697 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834c715-7ff0-4423-a5ec-6ed6864ecf81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7990cbe5c1641369d4de0d5c2d4c66b, user_id=0a9e7a884fcf4d21919fff85e78e4ad8, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:51.862501 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834c715-7ff0-4423-a5ec-6ed6864ecf81 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:51.876619 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1452/2934] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:13:51 2022] PUT /identity/v3/projects/b7990cbe5c1641369d4de0d5c2d4c66b/users/0a9e7a884fcf4d21919fff85e78e4ad8/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:51.881199 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5a6c5-f056-471b-b929-5d14c39e3b8e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:51.891324 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5a6c5-f056-471b-b929-5d14c39e3b8e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:51.947491 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a74c7-5a5d-4bf2-9983-ca757fe643a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:51.948117 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a74c7-5a5d-4bf2-9983-ca757fe643a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:51.948397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a74c7-5a5d-4bf2-9983-ca757fe643a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:51.948674 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a74c7-5a5d-4bf2-9983-ca757fe643a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:51.949367 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a74c7-5a5d-4bf2-9983-ca757fe643a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:51.951055 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a74c7-5a5d-4bf2-9983-ca757fe643a6 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:51.960794 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1483/2935] 10.222.0.83 () {58 vars in 1122 bytes} [Mon Aug 1 16:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:13:51.964950 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834c715-7ff0-4423-a5ec-6ed6864ecf81 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:51.975948 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834c715-7ff0-4423-a5ec-6ed6864ecf81 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:51.980693 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc0057-c978-4e8f-a85b-aa9f7387b8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:51.981367 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc0057-c978-4e8f-a85b-aa9f7387b8bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:51.981652 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc0057-c978-4e8f-a85b-aa9f7387b8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:51.981969 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc0057-c978-4e8f-a85b-aa9f7387b8bb admin admin] PATH_INFO: `/v3/projects/b7990cbe5c1641369d4de0d5c2d4c66b/users/0a9e7a884fcf4d21919fff85e78e4ad8/roles/7bdfa43dc1d046f7965ffe52b74bc58f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:51.984046 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc0057-c978-4e8f-a85b-aa9f7387b8bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7990cbe5c1641369d4de0d5c2d4c66b, user_id=0a9e7a884fcf4d21919fff85e78e4ad8, role_id=7bdfa43dc1d046f7965ffe52b74bc58f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:51.986827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc0057-c978-4e8f-a85b-aa9f7387b8bb admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:51.998618 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1453/2936] 10.222.0.83 () {60 vars in 1483 bytes} [Mon Aug 1 16:13:51 2022] PUT /identity/v3/projects/b7990cbe5c1641369d4de0d5c2d4c66b/users/0a9e7a884fcf4d21919fff85e78e4ad8/roles/7bdfa43dc1d046f7965ffe52b74bc58f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:52.008871 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1ee16-dc43-4239-9c83-732f6aa1f8fb None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:52.009183 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1ee16-dc43-4239-9c83-732f6aa1f8fb None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:52.009480 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1ee16-dc43-4239-9c83-732f6aa1f8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:52.037687 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a6b1ee16-dc43-4239-9c83-732f6aa1f8fb None None] MFA Rules not processed for user `0a9e7a884fcf4d21919fff85e78e4ad8`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:52.038915 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a6b1ee16-dc43-4239-9c83-732f6aa1f8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:52.113681 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1484/2937] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:52.125210 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc0057-c978-4e8f-a85b-aa9f7387b8bb admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:52.127097 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-cbdc0057-c978-4e8f-a85b-aa9f7387b8bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:52.180953 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:52.185309 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc0057-c978-4e8f-a85b-aa9f7387b8bb admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:52.189769 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4cb2b9-73b6-4d4a-a755-9da6615b3890 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54b570990794801a5e45918845569b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:52.190437 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cb2b9-73b6-4d4a-a755-9da6615b3890 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:52.191487 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cb2b9-73b6-4d4a-a755-9da6615b3890 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:52.191487 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cb2b9-73b6-4d4a-a755-9da6615b3890 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:52.200361 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4cb2b9-73b6-4d4a-a755-9da6615b3890 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:52.201807 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4cb2b9-73b6-4d4a-a755-9da6615b3890 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:52.211238 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1454/2938] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:13:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:13:52.351884 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1ee16-dc43-4239-9c83-732f6aa1f8fb None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:52.413509 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:52.420382 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1ee16-dc43-4239-9c83-732f6aa1f8fb None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:52.425432 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ef12b-ab83-4ab3-b44b-b27361c7eb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:52.426227 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ef12b-ab83-4ab3-b44b-b27361c7eb76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:52.426516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ef12b-ab83-4ab3-b44b-b27361c7eb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:52.426771 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ef12b-ab83-4ab3-b44b-b27361c7eb76 admin admin] PATH_INFO: `/v3/users/9a20d6a034c2449daaf05da97b2feb5e` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:52.427916 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ef12b-ab83-4ab3-b44b-b27361c7eb76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a20d6a034c2449daaf05da97b2feb5e)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:52.429855 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ef12b-ab83-4ab3-b44b-b27361c7eb76 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:52.476344 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5c1ef12b-ab83-4ab3-b44b-b27361c7eb76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a20d6a034c2449daaf05da97b2feb5e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:52.477000 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5c1ef12b-ab83-4ab3-b44b-b27361c7eb76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a20d6a034c2449daaf05da97b2feb5e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:52.484581 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5c1ef12b-ab83-4ab3-b44b-b27361c7eb76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a20d6a034c2449daaf05da97b2feb5e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:52.496913 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5c1ef12b-ab83-4ab3-b44b-b27361c7eb76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a20d6a034c2449daaf05da97b2feb5e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:52.503859 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5c1ef12b-ab83-4ab3-b44b-b27361c7eb76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a20d6a034c2449daaf05da97b2feb5e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:52.509745 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1485/2939] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:13:52 2022] DELETE /identity/v3/users/9a20d6a034c2449daaf05da97b2feb5e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:52.514338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673416c-fb3d-4408-8f53-b8b2f8945231 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:52.514629 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673416c-fb3d-4408-8f53-b8b2f8945231 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:52.514908 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673416c-fb3d-4408-8f53-b8b2f8945231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:52.534552 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7673416c-fb3d-4408-8f53-b8b2f8945231 None None] MFA Rules not processed for user `01b3cc95b90b46cf85a902fea655d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:13:52.535770 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7673416c-fb3d-4408-8f53-b8b2f8945231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:52.595461 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1455/2940] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:13:52.606528 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ef12b-ab83-4ab3-b44b-b27361c7eb76 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:52.607901 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5c1ef12b-ab83-4ab3-b44b-b27361c7eb76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:52.665035 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ef12b-ab83-4ab3-b44b-b27361c7eb76 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:52.668700 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab2ede-0aa7-4b44-8ff4-18191f422b74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:52.669318 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab2ede-0aa7-4b44-8ff4-18191f422b74 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:52.669603 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab2ede-0aa7-4b44-8ff4-18191f422b74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:52.669879 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab2ede-0aa7-4b44-8ff4-18191f422b74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:52.678974 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab2ede-0aa7-4b44-8ff4-18191f422b74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:52.680392 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab2ede-0aa7-4b44-8ff4-18191f422b74 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:52.690050 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1486/2941] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:13:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:13:52.962905 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673416c-fb3d-4408-8f53-b8b2f8945231 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:52.964427 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7673416c-fb3d-4408-8f53-b8b2f8945231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:52.976934 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673416c-fb3d-4408-8f53-b8b2f8945231 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:52.982278 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06655f-c195-4e8c-b44f-52956df4054a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:52.983079 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06655f-c195-4e8c-b44f-52956df4054a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:52.983594 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06655f-c195-4e8c-b44f-52956df4054a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:52.983942 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06655f-c195-4e8c-b44f-52956df4054a admin admin] PATH_INFO: `/v3/projects/f6b67ce7dcce4825b2b4747425f19441` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:52.985280 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06655f-c195-4e8c-b44f-52956df4054a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6b67ce7dcce4825b2b4747425f19441)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:52.987705 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06655f-c195-4e8c-b44f-52956df4054a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:53.038595 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6a06655f-c195-4e8c-b44f-52956df4054a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6b67ce7dcce4825b2b4747425f19441', 'request_id': 'req-6a06655f-c195-4e8c-b44f-52956df4054a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:53.046270 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6a06655f-c195-4e8c-b44f-52956df4054a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6b67ce7dcce4825b2b4747425f19441', 'request_id': 'req-6a06655f-c195-4e8c-b44f-52956df4054a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:53.054597 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6a06655f-c195-4e8c-b44f-52956df4054a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6b67ce7dcce4825b2b4747425f19441', 'request_id': 'req-6a06655f-c195-4e8c-b44f-52956df4054a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:53.068651 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6a06655f-c195-4e8c-b44f-52956df4054a admin admin] The token cache is being invalidate because project f6b67ce7dcce4825b2b4747425f19441 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:13:53.068932 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6a06655f-c195-4e8c-b44f-52956df4054a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:53.069577 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6a06655f-c195-4e8c-b44f-52956df4054a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:53.071354 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1456/2942] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:13:52 2022] DELETE /identity/v3/projects/f6b67ce7dcce4825b2b4747425f19441 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:53.075017 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab2ede-0aa7-4b44-8ff4-18191f422b74 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:53.084917 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab2ede-0aa7-4b44-8ff4-18191f422b74 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:53.144403 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa8294-8e6d-4db3-9818-3ef8bce5b16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:53.144965 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa8294-8e6d-4db3-9818-3ef8bce5b16d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:53.145248 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa8294-8e6d-4db3-9818-3ef8bce5b16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:53.145519 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa8294-8e6d-4db3-9818-3ef8bce5b16d admin admin] PATH_INFO: `/v3/users/0a9e7a884fcf4d21919fff85e78e4ad8` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:53.146742 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa8294-8e6d-4db3-9818-3ef8bce5b16d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a9e7a884fcf4d21919fff85e78e4ad8)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:53.148649 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa8294-8e6d-4db3-9818-3ef8bce5b16d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:53.168753 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:53.195513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2faa8294-8e6d-4db3-9818-3ef8bce5b16d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a9e7a884fcf4d21919fff85e78e4ad8'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:53.196168 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2faa8294-8e6d-4db3-9818-3ef8bce5b16d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a9e7a884fcf4d21919fff85e78e4ad8'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:53.203381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2faa8294-8e6d-4db3-9818-3ef8bce5b16d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a9e7a884fcf4d21919fff85e78e4ad8'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:53.215266 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2faa8294-8e6d-4db3-9818-3ef8bce5b16d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a9e7a884fcf4d21919fff85e78e4ad8'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:53.222171 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2faa8294-8e6d-4db3-9818-3ef8bce5b16d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a9e7a884fcf4d21919fff85e78e4ad8'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:53.228152 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1487/2943] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:13:53 2022] DELETE /identity/v3/users/0a9e7a884fcf4d21919fff85e78e4ad8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:13:53.756862 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06655f-c195-4e8c-b44f-52956df4054a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:13:53.758350 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6a06655f-c195-4e8c-b44f-52956df4054a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:13:53.771353 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06655f-c195-4e8c-b44f-52956df4054a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:13:53.830127 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a927a8e-453c-4505-aa19-66b7595acb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:13:53.830780 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a927a8e-453c-4505-aa19-66b7595acb2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:13:53.831062 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a927a8e-453c-4505-aa19-66b7595acb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:13:53.831338 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a927a8e-453c-4505-aa19-66b7595acb2f admin admin] PATH_INFO: `/v3/projects/b7990cbe5c1641369d4de0d5c2d4c66b` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:13:53.832476 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a927a8e-453c-4505-aa19-66b7595acb2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7990cbe5c1641369d4de0d5c2d4c66b)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:13:53.834980 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a927a8e-453c-4505-aa19-66b7595acb2f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:13:53.852389 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:13:53.883329 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6a927a8e-453c-4505-aa19-66b7595acb2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7990cbe5c1641369d4de0d5c2d4c66b', 'request_id': 'req-6a927a8e-453c-4505-aa19-66b7595acb2f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:53.889662 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6a927a8e-453c-4505-aa19-66b7595acb2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7990cbe5c1641369d4de0d5c2d4c66b', 'request_id': 'req-6a927a8e-453c-4505-aa19-66b7595acb2f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:53.895203 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6a927a8e-453c-4505-aa19-66b7595acb2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7990cbe5c1641369d4de0d5c2d4c66b', 'request_id': 'req-6a927a8e-453c-4505-aa19-66b7595acb2f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:53.908146 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6a927a8e-453c-4505-aa19-66b7595acb2f admin admin] The token cache is being invalidate because project b7990cbe5c1641369d4de0d5c2d4c66b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:13:53.908466 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6a927a8e-453c-4505-aa19-66b7595acb2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:53.909091 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6a927a8e-453c-4505-aa19-66b7595acb2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:13:53.911245 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1457/2944] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:13:53 2022] DELETE /identity/v3/projects/b7990cbe5c1641369d4de0d5c2d4c66b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:14:03.224633 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:DELETE=8,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:14:03.905075 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=17,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:15:28.047574 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bba25e-e0df-4be1-aba3-c22100ac3ca7 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:15:28.048147 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bba25e-e0df-4be1-aba3-c22100ac3ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:15:28.048442 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bba25e-e0df-4be1-aba3-c22100ac3ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:15:28.075592 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d8bba25e-e0df-4be1-aba3-c22100ac3ca7 None None] MFA Rules not processed for user `c1e71f7596824b7e9c7a88a22a6664c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:15:28.076811 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d8bba25e-e0df-4be1-aba3-c22100ac3ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:15:28.134397 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1488/2945] 10.222.0.83 () {58 vars in 958 bytes} [Mon Aug 1 16:15:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:15:28.896185 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a927a8e-453c-4505-aa19-66b7595acb2f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:15:28.898318 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6a927a8e-453c-4505-aa19-66b7595acb2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:15:28.973937 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a927a8e-453c-4505-aa19-66b7595acb2f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:15:28.978992 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da47bdb-53b9-4290-93b9-3f9efca5a38a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:15:28.979716 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da47bdb-53b9-4290-93b9-3f9efca5a38a service placement] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:15:28.980001 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da47bdb-53b9-4290-93b9-3f9efca5a38a service placement] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:15:28.980276 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da47bdb-53b9-4290-93b9-3f9efca5a38a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:15:28.982062 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4da47bdb-53b9-4290-93b9-3f9efca5a38a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:15:28.993643 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da47bdb-53b9-4290-93b9-3f9efca5a38a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:15:28.995597 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da47bdb-53b9-4290-93b9-3f9efca5a38a service placement] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:15:29.005497 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1458/2946] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:15:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:15:31.701598 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bba25e-e0df-4be1-aba3-c22100ac3ca7 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:15:31.704473 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d8bba25e-e0df-4be1-aba3-c22100ac3ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:15:31.801950 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bba25e-e0df-4be1-aba3-c22100ac3ca7 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:15:31.808094 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85cb4c-f731-4679-831b-d6c430cb9fe2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:15:31.808772 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85cb4c-f731-4679-831b-d6c430cb9fe2 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:15:31.809054 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85cb4c-f731-4679-831b-d6c430cb9fe2 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:15:31.809327 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85cb4c-f731-4679-831b-d6c430cb9fe2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:15:31.811243 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-bf85cb4c-f731-4679-831b-d6c430cb9fe2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:15:31.827188 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85cb4c-f731-4679-831b-d6c430cb9fe2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:15:31.828823 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85cb4c-f731-4679-831b-d6c430cb9fe2 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:15:31.840833 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1489/2947] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:15:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:15:39.001200 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=32 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:15:39.523589 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da47bdb-53b9-4290-93b9-3f9efca5a38a service placement] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:15:39.525002 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4da47bdb-53b9-4290-93b9-3f9efca5a38a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:15:39.540237 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da47bdb-53b9-4290-93b9-3f9efca5a38a service placement] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:15:39.550168 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd398898-8b02-42e0-80d9-cfd03be7f5d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:15:39.550819 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd398898-8b02-42e0-80d9-cfd03be7f5d1 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:15:39.551097 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd398898-8b02-42e0-80d9-cfd03be7f5d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:15:39.551369 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd398898-8b02-42e0-80d9-cfd03be7f5d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:15:39.553092 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-fd398898-8b02-42e0-80d9-cfd03be7f5d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:15:39.565390 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd398898-8b02-42e0-80d9-cfd03be7f5d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:15:39.567016 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd398898-8b02-42e0-80d9-cfd03be7f5d1 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:15:39.576000 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1459/2948] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:15:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:15:41.835322 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=66 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:15:42.093127 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85cb4c-f731-4679-831b-d6c430cb9fe2 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:15:42.111764 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85cb4c-f731-4679-831b-d6c430cb9fe2 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:15:42.117491 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bdf3e-be61-4858-b843-04472ee0da44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:15:42.118146 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bdf3e-be61-4858-b843-04472ee0da44 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:15:42.118435 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bdf3e-be61-4858-b843-04472ee0da44 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:15:42.118710 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bdf3e-be61-4858-b843-04472ee0da44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:15:42.121532 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2a2bdf3e-be61-4858-b843-04472ee0da44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:15:42.134446 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bdf3e-be61-4858-b843-04472ee0da44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:15:42.136036 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bdf3e-be61-4858-b843-04472ee0da44 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:15:42.148510 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1490/2949] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:15:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:15:46.242823 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd398898-8b02-42e0-80d9-cfd03be7f5d1 service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:15:46.255763 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd398898-8b02-42e0-80d9-cfd03be7f5d1 service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:15:46.262137 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125640d-3682-48f3-a3ce-543a9fbaaf3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dc41ffc41c47fdbeec06110d04610c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:15:46.263465 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125640d-3682-48f3-a3ce-543a9fbaaf3a service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:15:46.263465 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125640d-3682-48f3-a3ce-543a9fbaaf3a service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:15:46.263930 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125640d-3682-48f3-a3ce-543a9fbaaf3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:15:46.265732 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0125640d-3682-48f3-a3ce-543a9fbaaf3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:15:46.279512 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125640d-3682-48f3-a3ce-543a9fbaaf3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:15:46.281236 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125640d-3682-48f3-a3ce-543a9fbaaf3a service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:15:46.299082 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1460/2950] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:15:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:15:52.143364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=6 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:15:56.293642 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=12 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:16:25.265323 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bdf3e-be61-4858-b843-04472ee0da44 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:16:25.266888 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2a2bdf3e-be61-4858-b843-04472ee0da44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:16:25.336047 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bdf3e-be61-4858-b843-04472ee0da44 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:16:25.340095 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a8003-1788-442f-8d2e-77a72dc9ce12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:16:25.340716 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a8003-1788-442f-8d2e-77a72dc9ce12 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:16:25.340999 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a8003-1788-442f-8d2e-77a72dc9ce12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:16:25.341275 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a8003-1788-442f-8d2e-77a72dc9ce12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:16:25.343094 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-db5a8003-1788-442f-8d2e-77a72dc9ce12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:16:25.356853 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a8003-1788-442f-8d2e-77a72dc9ce12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:16:25.358353 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a8003-1788-442f-8d2e-77a72dc9ce12 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:16:25.368078 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1491/2951] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:16:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:16:35.363847 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=30 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:17:18.573657 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125640d-3682-48f3-a3ce-543a9fbaaf3a service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:17:18.575781 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0125640d-3682-48f3-a3ce-543a9fbaaf3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:17:18.596844 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125640d-3682-48f3-a3ce-543a9fbaaf3a service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:17:18.603778 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4679ff-f79e-46e5-b6b4-b210e5c26384 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:17:18.604646 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4679ff-f79e-46e5-b6b4-b210e5c26384 service placement] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:17:18.604936 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4679ff-f79e-46e5-b6b4-b210e5c26384 service placement] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:17:18.605214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4679ff-f79e-46e5-b6b4-b210e5c26384 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:17:18.607132 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4b4679ff-f79e-46e5-b6b4-b210e5c26384 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:17:18.620088 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4679ff-f79e-46e5-b6b4-b210e5c26384 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:17:18.621935 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4679ff-f79e-46e5-b6b4-b210e5c26384 service placement] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:17:18.632903 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1461/2952] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:17:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:17:18.644513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a8003-1788-442f-8d2e-77a72dc9ce12 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:17:18.656770 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a8003-1788-442f-8d2e-77a72dc9ce12 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:17:18.660881 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9f1d1-6b53-4dc9-99ec-4a375f52bbbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:17:18.661722 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9f1d1-6b53-4dc9-99ec-4a375f52bbbe service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:17:18.662049 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9f1d1-6b53-4dc9-99ec-4a375f52bbbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:17:18.662333 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9f1d1-6b53-4dc9-99ec-4a375f52bbbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:17:18.664391 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ceb9f1d1-6b53-4dc9-99ec-4a375f52bbbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:17:18.678934 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb9f1d1-6b53-4dc9-99ec-4a375f52bbbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:17:18.680381 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb9f1d1-6b53-4dc9-99ec-4a375f52bbbe service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:17:18.691748 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1492/2953] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:17:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:17:19.882845 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4679ff-f79e-46e5-b6b4-b210e5c26384 service placement] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:17:19.893221 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4679ff-f79e-46e5-b6b4-b210e5c26384 service placement] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:17:19.896851 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd014f-7e8a-494e-9b1b-942fa6f76076 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:17:19.898041 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd014f-7e8a-494e-9b1b-942fa6f76076 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:17:19.898326 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd014f-7e8a-494e-9b1b-942fa6f76076 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:17:19.898603 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd014f-7e8a-494e-9b1b-942fa6f76076 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:17:19.900477 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1dcd014f-7e8a-494e-9b1b-942fa6f76076 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:17:19.917259 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd014f-7e8a-494e-9b1b-942fa6f76076 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:17:19.921452 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd014f-7e8a-494e-9b1b-942fa6f76076 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:17:19.932956 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1462/2954] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:17:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:17:28.686679 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=6 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:17:29.926951 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=12 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:17:31.136379 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9f1d1-6b53-4dc9-99ec-4a375f52bbbe service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:17:31.138414 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ceb9f1d1-6b53-4dc9-99ec-4a375f52bbbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:17:31.149474 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9f1d1-6b53-4dc9-99ec-4a375f52bbbe service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:17:31.153436 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a9ed5-9e23-4e6d-813e-9607d87cd5fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76311c7173b442c0af2c745e3e70a07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:17:31.154085 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a9ed5-9e23-4e6d-813e-9607d87cd5fd service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:17:31.154364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a9ed5-9e23-4e6d-813e-9607d87cd5fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:17:31.154634 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a9ed5-9e23-4e6d-813e-9607d87cd5fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:17:31.156409 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7e8a9ed5-9e23-4e6d-813e-9607d87cd5fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:17:31.225121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a9ed5-9e23-4e6d-813e-9607d87cd5fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:17:31.226603 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a9ed5-9e23-4e6d-813e-9607d87cd5fd service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:17:31.236661 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1493/2955] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:17:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:17:34.357973 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd014f-7e8a-494e-9b1b-942fa6f76076 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:17:34.370197 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd014f-7e8a-494e-9b1b-942fa6f76076 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:17:34.375418 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dff3b6-8938-4a27-9e0b-f9441dca3f4c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:17:34.376058 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dff3b6-8938-4a27-9e0b-f9441dca3f4c service placement] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:17:34.376343 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dff3b6-8938-4a27-9e0b-f9441dca3f4c service placement] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:17:34.376614 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dff3b6-8938-4a27-9e0b-f9441dca3f4c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:17:34.378563 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-18dff3b6-8938-4a27-9e0b-f9441dca3f4c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:17:34.391236 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dff3b6-8938-4a27-9e0b-f9441dca3f4c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:17:34.392794 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dff3b6-8938-4a27-9e0b-f9441dca3f4c service placement] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:17:34.404381 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1463/2956] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:17:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:17:37.583516 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a9ed5-9e23-4e6d-813e-9607d87cd5fd service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:17:37.607445 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a9ed5-9e23-4e6d-813e-9607d87cd5fd service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:17:37.615213 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf421e-b1a6-4ec3-bf8f-e10c524ba1ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:17:37.615935 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf421e-b1a6-4ec3-bf8f-e10c524ba1ea service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:17:37.616311 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf421e-b1a6-4ec3-bf8f-e10c524ba1ea service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:17:37.616662 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf421e-b1a6-4ec3-bf8f-e10c524ba1ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:17:37.618946 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-bfcf421e-b1a6-4ec3-bf8f-e10c524ba1ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:17:37.638364 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf421e-b1a6-4ec3-bf8f-e10c524ba1ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:17:37.640005 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf421e-b1a6-4ec3-bf8f-e10c524ba1ea service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:17:37.646314 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dff3b6-8938-4a27-9e0b-f9441dca3f4c service placement] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:17:37.648288 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-18dff3b6-8938-4a27-9e0b-f9441dca3f4c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:17:37.652538 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1494/2957] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:17:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:17:37.677605 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dff3b6-8938-4a27-9e0b-f9441dca3f4c service placement] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:17:37.683321 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df71992-9030-409d-85fd-eda621fbf84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:17:37.684214 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df71992-9030-409d-85fd-eda621fbf84c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:17:37.684521 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df71992-9030-409d-85fd-eda621fbf84c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:17:37.684799 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df71992-9030-409d-85fd-eda621fbf84c admin admin] PATH_INFO: `/v3/users/c1e71f7596824b7e9c7a88a22a6664c3` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:17:37.686753 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df71992-9030-409d-85fd-eda621fbf84c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1e71f7596824b7e9c7a88a22a6664c3)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:17:37.689035 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df71992-9030-409d-85fd-eda621fbf84c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:17:37.747818 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8df71992-9030-409d-85fd-eda621fbf84c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1e71f7596824b7e9c7a88a22a6664c3'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:17:37.748495 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8df71992-9030-409d-85fd-eda621fbf84c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1e71f7596824b7e9c7a88a22a6664c3'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:17:37.756664 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8df71992-9030-409d-85fd-eda621fbf84c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1e71f7596824b7e9c7a88a22a6664c3'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:17:37.769827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8df71992-9030-409d-85fd-eda621fbf84c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1e71f7596824b7e9c7a88a22a6664c3'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:17:37.770493 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8df71992-9030-409d-85fd-eda621fbf84c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1e71f7596824b7e9c7a88a22a6664c3'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:17:37.783218 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1464/2958] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:17:37 2022] DELETE /identity/v3/users/c1e71f7596824b7e9c7a88a22a6664c3 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:17:37.970585 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf421e-b1a6-4ec3-bf8f-e10c524ba1ea service placement] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:17:37.981734 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf421e-b1a6-4ec3-bf8f-e10c524ba1ea service placement] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:17:38.039375 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f58a6-2c4f-4e1a-b1da-59c00ee6846f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:17:38.040028 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f58a6-2c4f-4e1a-b1da-59c00ee6846f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:17:38.040308 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f58a6-2c4f-4e1a-b1da-59c00ee6846f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:17:38.040586 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f58a6-2c4f-4e1a-b1da-59c00ee6846f admin admin] PATH_INFO: `/v3/projects/c12ec1d14cb44902bed525172feed2f8` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:17:38.042481 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f58a6-2c4f-4e1a-b1da-59c00ee6846f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c12ec1d14cb44902bed525172feed2f8)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:17:38.044395 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f58a6-2c4f-4e1a-b1da-59c00ee6846f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:17:38.098738 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-831f58a6-2c4f-4e1a-b1da-59c00ee6846f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c12ec1d14cb44902bed525172feed2f8', 'request_id': 'req-831f58a6-2c4f-4e1a-b1da-59c00ee6846f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:17:38.104279 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-831f58a6-2c4f-4e1a-b1da-59c00ee6846f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c12ec1d14cb44902bed525172feed2f8', 'request_id': 'req-831f58a6-2c4f-4e1a-b1da-59c00ee6846f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:17:38.110393 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-831f58a6-2c4f-4e1a-b1da-59c00ee6846f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c12ec1d14cb44902bed525172feed2f8', 'request_id': 'req-831f58a6-2c4f-4e1a-b1da-59c00ee6846f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:17:38.123401 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-831f58a6-2c4f-4e1a-b1da-59c00ee6846f admin admin] The token cache is being invalidate because project c12ec1d14cb44902bed525172feed2f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:17:38.123689 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-831f58a6-2c4f-4e1a-b1da-59c00ee6846f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:17:38.124457 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-831f58a6-2c4f-4e1a-b1da-59c00ee6846f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:17:38.126047 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1495/2959] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:17:37 2022] DELETE /identity/v3/projects/c12ec1d14cb44902bed525172feed2f8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:17:38.130086 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df71992-9030-409d-85fd-eda621fbf84c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:17:38.131514 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8df71992-9030-409d-85fd-eda621fbf84c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:17:38.148790 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df71992-9030-409d-85fd-eda621fbf84c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:17:38.206619 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce7b67-d74e-424e-9781-f85121883097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:17:38.207260 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce7b67-d74e-424e-9781-f85121883097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:17:38.207541 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce7b67-d74e-424e-9781-f85121883097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:17:38.207814 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce7b67-d74e-424e-9781-f85121883097 admin admin] PATH_INFO: `/v3/users/b695d7a8c77b4250921140b894acae0d` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:17:38.208985 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce7b67-d74e-424e-9781-f85121883097 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b695d7a8c77b4250921140b894acae0d)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:17:38.210946 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce7b67-d74e-424e-9781-f85121883097 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:17:38.260510 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-74ce7b67-d74e-424e-9781-f85121883097 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b695d7a8c77b4250921140b894acae0d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:17:38.261440 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-74ce7b67-d74e-424e-9781-f85121883097 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b695d7a8c77b4250921140b894acae0d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:17:38.268527 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-74ce7b67-d74e-424e-9781-f85121883097 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b695d7a8c77b4250921140b894acae0d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:17:38.273857 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:17:38.282773 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-74ce7b67-d74e-424e-9781-f85121883097 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b695d7a8c77b4250921140b894acae0d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:17:38.283598 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-74ce7b67-d74e-424e-9781-f85121883097 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b695d7a8c77b4250921140b894acae0d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:17:38.295827 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1465/2960] 10.222.0.83 () {58 vars in 1224 bytes} [Mon Aug 1 16:17:38 2022] DELETE /identity/v3/users/b695d7a8c77b4250921140b894acae0d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:17:38.474338 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f58a6-2c4f-4e1a-b1da-59c00ee6846f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:17:38.476121 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-831f58a6-2c4f-4e1a-b1da-59c00ee6846f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:17:38.496832 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f58a6-2c4f-4e1a-b1da-59c00ee6846f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:17:38.509040 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:17:38.560704 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc3ebe-95c8-47f4-b8b5-4a87dba688ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b3cc95b90b46cf85a902fea655d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0cb293301d4dcdbc4412b1e11a284f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:17:38.561372 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc3ebe-95c8-47f4-b8b5-4a87dba688ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:17:38.561653 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc3ebe-95c8-47f4-b8b5-4a87dba688ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:17:38.562043 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc3ebe-95c8-47f4-b8b5-4a87dba688ce admin admin] PATH_INFO: `/v3/projects/3e3600e9a58647e398779fba36146127` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:17:38.563196 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc3ebe-95c8-47f4-b8b5-4a87dba688ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e3600e9a58647e398779fba36146127)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:17:38.565096 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc3ebe-95c8-47f4-b8b5-4a87dba688ce admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:17:38.611600 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a7dc3ebe-95c8-47f4-b8b5-4a87dba688ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e3600e9a58647e398779fba36146127', 'request_id': 'req-a7dc3ebe-95c8-47f4-b8b5-4a87dba688ce'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:17:38.617026 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a7dc3ebe-95c8-47f4-b8b5-4a87dba688ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e3600e9a58647e398779fba36146127', 'request_id': 'req-a7dc3ebe-95c8-47f4-b8b5-4a87dba688ce'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:17:38.624047 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a7dc3ebe-95c8-47f4-b8b5-4a87dba688ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e3600e9a58647e398779fba36146127', 'request_id': 'req-a7dc3ebe-95c8-47f4-b8b5-4a87dba688ce'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:17:38.636604 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a7dc3ebe-95c8-47f4-b8b5-4a87dba688ce admin admin] The token cache is being invalidate because project 3e3600e9a58647e398779fba36146127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:17:38.636893 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a7dc3ebe-95c8-47f4-b8b5-4a87dba688ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:17:38.637526 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a7dc3ebe-95c8-47f4-b8b5-4a87dba688ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:17:38.639122 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1496/2961] 10.222.0.83 () {58 vars in 1233 bytes} [Mon Aug 1 16:17:38 2022] DELETE /identity/v3/projects/3e3600e9a58647e398779fba36146127 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:17:48.292340 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=9,keystone:DELETE=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:17:48.633513 dv-cinder-847536-3-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:22:20.532342 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce7b67-d74e-424e-9781-f85121883097 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:22:20.533792 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-74ce7b67-d74e-424e-9781-f85121883097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:22:20.665477 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce7b67-d74e-424e-9781-f85121883097 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:22:20.736944 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856c7b8-f57d-4a03-95b5-f19f51c08b02 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef57a9394b114eb3ab499133b5d40781', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f4e27cc50bd4f9f8ced4cc544ab3064', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:22:20.737570 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856c7b8-f57d-4a03-95b5-f19f51c08b02 service placement] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:22:20.737853 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856c7b8-f57d-4a03-95b5-f19f51c08b02 service placement] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:22:20.738155 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856c7b8-f57d-4a03-95b5-f19f51c08b02 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:22:20.741387 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1856c7b8-f57d-4a03-95b5-f19f51c08b02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:22:20.753378 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856c7b8-f57d-4a03-95b5-f19f51c08b02 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:22:20.755020 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856c7b8-f57d-4a03-95b5-f19f51c08b02 service placement] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:22:20.766647 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1466/2962] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Aug 1 16:22:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:22:30.761907 dv-cinder-847536-3-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=80 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}